# groonga - An open-source fulltext search engine and column store.

- 日本語版はこちら

### Navigation

- index
- next |
- previous |
- groonga v2.0.5-154-g8f85693 documentation »
- 8. リファレンスマニュアル »
- 8.4. コマンド »

# 8.4.17. select

## 8.4.17.1. Summary

`select` searches records that are matched to specified conditions from a table and then outputs them.

`select` is the most important command in groonga. You need to understand `select` to use the full power of groonga.

## 8.4.17.2. Syntax

`select` has many parameters. The required parameter is only `table` and others are optional:

    select table
           [match_columns=null]
           [query=null]
           [filter=null]
           [scorer=null]
           [sortby=null]
           [output_columns="_id, _key, *"]
           [offset=0]
           [limit=10]
           [drilldown=null]
           [drilldown_sortby=null]
           [drilldown_output_columns=null]
           [drilldown_offset=0]
           [drilldown_limit=10]
           [cache=yes]
           [match_escalation_threshold=0]
           [query_expansion=null]
           [query_flags=ALLOW_PRAGMA|ALLOW_COLUMN|ALLOW_UPDATE|NONE]

## 8.4.17.3. Usage

Let's learn about `select` usage with examples. This section shows many popular usages.

Here are a schema definition and sample data to show usage.

Execution example:

    table_create Entries TABLE_HASH_KEY ShortText
    # [[0, 1337566253.89858, 0.000355720520019531], true]
    column_create Entries content COLUMN_SCALAR Text
    # [[0, 1337566253.89858, 0.000355720520019531], true]
    column_create Entries n_likes COLUMN_SCALAR UInt32
    # [[0, 1337566253.89858, 0.000355720520019531], true]
    table_create Terms TABLE_PAT_KEY|KEY_NORMALIZE ShortText --default_tokenizer TokenBigram
    # [[0, 1337566253.89858, 0.000355720520019531], true]
    column_create Terms entries_key_index COLUMN_INDEX|WITH_POSITION Entries _key
    # [[0, 1337566253.89858, 0.000355720520019531], true]
    column_create Terms entries_content_index COLUMN_INDEX|WITH_POSITION Entries content
    # [[0, 1337566253.89858, 0.000355720520019531], true]
    load --table Entries
    [
    {"_key":    "The first post!",
     "content": "Welcome! This is my first post!",
     "n_likes": 5},
    {"_key":    "Groonga",
     "content": "I started to use groonga. It's very fast!",
     "n_likes": 10},
    {"_key":    "Mroonga",
     "content": "I also started to use mroonga. It's also very fast! Really fast!",
     "n_likes": 15},
    {"_key":    "Good-bye Senna",
     "content": "I migrated all Senna system!",
     "n_likes": 3},
    {"_key":    "Good-bye Tritonn",
     "content": "I also migrated all Tritonn system!",
     "n_likes": 3}
    ]
    # [[0, 1337566253.89858, 0.000355720520019531], 5]

There is a table, `Entries`, for blog entries. An entry has title, content and the number of likes for the entry. Title is key of `Entries`. Content is value of `Entries.content` column. The number of likes is value of `Entries.n_likes` column.

`Entries._key` column and `Entries.content` column are indexed using `TokenBigram` tokenizer. So both `Entries._key` and `Entries.content` are fulltext search ready.

OK. The schema and data for examples are ready.

### 8.4.17.3.1. Simple usage

Here is the most simple usage with the above schema and data. It outputs all records in `Entries` table.

Execution example:

    select Entries
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         5
    #       ],
    #       [
    #         [
    #           "_id",
    #           "UInt32"
    #         ],
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "content",
    #           "Text"
    #         ],
    #         [
    #           "n_likes",
    #           "UInt32"
    #         ]
    #       ],
    #       [
    #         1,
    #         "The first post!",
    #         "Welcome! This is my first post!",
    #         5
    #       ],
    #       [
    #         2,
    #         "Groonga",
    #         "I started to use groonga. It's very fast!",
    #         10
    #       ],
    #       [
    #         3,
    #         "Mroonga",
    #         "I also started to use mroonga. It's also very fast! Really fast!",
    #         15
    #       ],
    #       [
    #         4,
    #         "Good-bye Senna",
    #         "I migrated all Senna system!",
    #         3
    #       ],
    #       [
    #         5,
    #         "Good-bye Tritonn",
    #         "I also migrated all Tritonn system!",
    #         3
    #       ]
    #     ]
    #   ]
    # ]

Why does the command output all records? There are two reasons. The first reason is that the command doesn't specify any search conditions. No search condition means all records are matched. The second reason is that the number of all records is 5. `select` command outputs 10 records at a maximum by default. There are only 5 records. It is less than 10. So the command outputs all records.

### 8.4.17.3.2. Search conditions

Search conditions are specified by `query` or `filter`. You can also specify both `query` and `filter`. It means that selected records must be matched against both `query` and `filter`.

#### 8.4.17.3.2.1. Search condition: `query`

`query` is designed for search box in Web page. Imagine a search box in google.com. You specify search conditions for `query` as space separated keywords. For example, `search engine` means a matched record should contain two words, `search` and `engine`.

Normally, `query` parameter is used for specifying fulltext search conditions. It can be used for non fulltext search conditions but `filter` is used for the propose.

`query` parameter is used with `match_columns` parameter when `query` parameter is used for specifying fulltext search conditions. `match_columns` specifies which columnes and indexes are matched against `query`.

Here is a simple `query` usage example.

Execution example:

    select Entries --match_columns content --query fast
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         2
    #       ],
    #       [
    #         [
    #           "_id",
    #           "UInt32"
    #         ],
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "content",
    #           "Text"
    #         ],
    #         [
    #           "n_likes",
    #           "UInt32"
    #         ]
    #       ],
    #       [
    #         2,
    #         "Groonga",
    #         "I started to use groonga. It's very fast!",
    #         10
    #       ],
    #       [
    #         3,
    #         "Mroonga",
    #         "I also started to use mroonga. It's also very fast! Really fast!",
    #         15
    #       ]
    #     ]
    #   ]
    # ]

The `select` command searches records that contain a word `fast` in `content` column value from `Entries` table.

`query` has query syntax but its deatils aren't described here. See Query syntax for datails.

#### 8.4.17.3.2.2. Search condition: `filter`

`filter` is designed for complex search conditions. You specify search conditions for `filter` as ECMAScript like syntax.

Here is a simple `filter` usage example.

Execution example:

    select Entries --filter 'content @ "fast" && _key == "Groonga"'
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         1
    #       ],
    #       [
    #         [
    #           "_id",
    #           "UInt32"
    #         ],
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "content",
    #           "Text"
    #         ],
    #         [
    #           "n_likes",
    #           "UInt32"
    #         ]
    #       ],
    #       [
    #         2,
    #         "Groonga",
    #         "I started to use groonga. It's very fast!",
    #         10
    #       ]
    #     ]
    #   ]
    # ]

The `select` command searches records that contain a word `fast` in `content` column value and has `Groonga` as `_key` from `Entries` table. There are three operators in the command, `@`, `&&` and `==`. `@` is fulltext search operator. `&&` and `==` are the same as ECMAScript. `&&` is logical AND operator and `==` is equality operator.

`filter` has more operators and syntax like grouping by `(...)` its deatils aren't described here. See Script syntax for datails.

### 8.4.17.3.3. Paging

You can specify range of outputted records by `offset` and `limit`. Here is an example to output only the 2nd record.

Execution example:

    select Entries --offset 1 --limit 1
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         5
    #       ],
    #       [
    #         [
    #           "_id",
    #           "UInt32"
    #         ],
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "content",
    #           "Text"
    #         ],
    #         [
    #           "n_likes",
    #           "UInt32"
    #         ]
    #       ],
    #       [
    #         2,
    #         "Groonga",
    #         "I started to use groonga. It's very fast!",
    #         10
    #       ]
    #     ]
    #   ]
    # ]

`offset` is zero-origin. `--offset 1` means output range is started from the 2nd record.

`limit` specifies the max number of output records. `--limit 1` means the number of output records is 1 at a maximium. If no records are matched, `select` command outputs no records.

### 8.4.17.3.4. The total number of records

You can use `--limit 0` to retrieve the total number of recrods without any contents of records.

Execution example:

    select Entries --limit 0
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         5
    #       ],
    #       [
    #         [
    #           "_id",
    #           "UInt32"
    #         ],
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "content",
    #           "Text"
    #         ],
    #         [
    #           "n_likes",
    #           "UInt32"
    #         ]
    #       ]
    #     ]
    #   ]
    # ]

`--limit 0` is also useful for retrieving only the number of matched records.

## 8.4.17.4. Parameters

This section describes all parameters. Parameters are categorized.

### 8.4.17.4.1. Required parameter

There is a required parameter, `table`.

#### 8.4.17.4.1.1. `table`

It specifies a table to be searched. `table` must be specified.

If nonexistent table is specified, an error is returned.

Execution example:

    select Nonexistent
    # [
    #   [
    #     -22,
    #     1337566253.89858,
    #     0.000355720520019531,
    #     "invalid table name: <Nonexistent>",
    #     [
    #       [
    #         "grn_select",
    #         "proc.c",
    #         542
    #       ]
    #     ]
    #   ]
    # ]

### 8.4.17.4.2. Search related parameters

There are search related parameters. Typically, `match_columns` and `query` parameters are used for implementing a search box. `filter` parameters is used for implementing complex search feature.

If both `query` and `filter` are specified, selected records must be matched against both `query` and `filter`. If both `query` and `filter` aren't specified, all records are selected.

#### 8.4.17.4.2.1. `match_columns`

It specifies the default target column for fulltext search by `query` parameter value. A target column for fulltext search can be specified in `query` parameter. The difference between `match_columns` and `query` is whether weight is supported or not. `match_columns` supports weight but `query` doesn't.

Weight is relative importance of target column. A higher weight target column gets more hit score rather than a lower weight target column when a record is matched by fulltext search. The default weight is 1.

Here is a simple `match_columns` usage example.

Execution example:

    select Entries --match_columns content --query fast --output_columns '_key, _score'
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         2
    #       ],
    #       [
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "_score",
    #           "Int32"
    #         ]
    #       ],
    #       [
    #         "Groonga",
    #         1
    #       ],
    #       [
    #         "Mroonga",
    #         2
    #       ]
    #     ]
    #   ]
    # ]

`--match_columns content` means the default target column for fulltext search is `content` column and its weight is 1. `--output_columns '_key, _score'` means that the `select` command outputs `_key` value and `_score` value for matched records.

Pay attention to `_score` value. `_score` value is the number of matched counts against `query` parameter value. In the example, `query` parameter value is `fast`. The fact that `_score` value is 1 means that `fast` appers in `content` column only once. The fact that `_score` value is 2 means that `fast` appears in `content` column twice.

To specify weight, `column * weight` syntax is used. Here is a weight usage example.

Execution example:

    select Entries --match_columns 'content * 2' --query fast --output_columns '_key, _score'
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         2
    #       ],
    #       [
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "_score",
    #           "Int32"
    #         ]
    #       ],
    #       [
    #         "Groonga",
    #         2
    #       ],
    #       [
    #         "Mroonga",
    #         4
    #       ]
    #     ]
    #   ]
    # ]

`--match_columns 'content * 2'` means the default target column for fulltext search is `content` column and its weight is 2.

Pay attention to `_score` value. `_score` value is doubled because weight is 2.

You can specify one or more columns as the default target columns for fulltext search. If one or more columns are specified, fulltext search is done for all columns and scores are accumulated. If one of the columns is matched against `query` parameter value, the record is treated as matched.

To specify one or more columns, `column1 * weight1 || column2 * weight2 || ...` syntax is used. `* weight` can be omitted. If it is omitted, 1 is used for weight. Here is a one or more columns usage example.

Execution example:

    select Entries --match_columns '_key * 10 || content' --query groonga --output_columns '_key, _score'
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         1
    #       ],
    #       [
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "_score",
    #           "Int32"
    #         ]
    #       ],
    #       [
    #         "Groonga",
    #         11
    #       ]
    #     ]
    #   ]
    # ]

`--match_columns '_key * 10 || content'` means the default target columns for fulltext search are `_key` and `content` columns and `_key` column's weight is 10 and `content` column's weight is 1. This weight allocation means `_key` column value is more important rather than `content` column value. In this example, title of blog entry is more important rather thatn content of blog entry.

#### 8.4.17.4.2.2. `query`

It specifies the query text. Normally, it is used for fulltext search with `match_columns` parameter. `query` parameter is designed for a fulltext search form in a Web page. A query text should be formatted in Query syntax. The syntax is similar to common search form like Google's search form. For example, `word1 word2` means that groonga searches records that contain both `word1` and `word2`. `word1 OR word2` means that groogna searches records that contain either `word1` or `word2`.

Here is a simple logical and search example.

Execution example:

    select Entries --match_columns content --query "fast groonga"
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         1
    #       ],
    #       [
    #         [
    #           "_id",
    #           "UInt32"
    #         ],
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "content",
    #           "Text"
    #         ],
    #         [
    #           "n_likes",
    #           "UInt32"
    #         ]
    #       ],
    #       [
    #         2,
    #         "Groonga",
    #         "I started to use groonga. It's very fast!",
    #         10
    #       ]
    #     ]
    #   ]
    # ]

The `select` command searches records that contain two words `fast` and `groonga` in `content` column value from `Entries` table.

Here is a simple logical or search example.

Execution example:

    select Entries --match_columns content --query "groonga OR mroonga"
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         2
    #       ],
    #       [
    #         [
    #           "_id",
    #           "UInt32"
    #         ],
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "content",
    #           "Text"
    #         ],
    #         [
    #           "n_likes",
    #           "UInt32"
    #         ]
    #       ],
    #       [
    #         2,
    #         "Groonga",
    #         "I started to use groonga. It's very fast!",
    #         10
    #       ],
    #       [
    #         3,
    #         "Mroonga",
    #         "I also started to use mroonga. It's also very fast! Really fast!",
    #         15
    #       ]
    #     ]
    #   ]
    # ]

The `select` command searches records that contain one of two words `groonga` or `mroonga` in `content` column value from `Entries` table.

See Query syntax for other syntax.

It can be used for not only fulltext search but also other conditions. For example, `column:value` means the value of `column` column is equal to `value`. `column:<value` means the value of `column` column is less than `value`.

Here is a simple equality operator search example.

Execution example:

    select Entries --query _key:Groonga
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         1
    #       ],
    #       [
    #         [
    #           "_id",
    #           "UInt32"
    #         ],
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "content",
    #           "Text"
    #         ],
    #         [
    #           "n_likes",
    #           "UInt32"
    #         ]
    #       ],
    #       [
    #         2,
    #         "Groonga",
    #         "I started to use groonga. It's very fast!",
    #         10
    #       ]
    #     ]
    #   ]
    # ]

The `select` command searches records that `_key` column value is `Groonga` from `Entries` table.

Here is a simple less than operator search example.

Execution example:

    select Entries --query n_likes:<11
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         4
    #       ],
    #       [
    #         [
    #           "_id",
    #           "UInt32"
    #         ],
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "content",
    #           "Text"
    #         ],
    #         [
    #           "n_likes",
    #           "UInt32"
    #         ]
    #       ],
    #       [
    #         1,
    #         "The first post!",
    #         "Welcome! This is my first post!",
    #         5
    #       ],
    #       [
    #         2,
    #         "Groonga",
    #         "I started to use groonga. It's very fast!",
    #         10
    #       ],
    #       [
    #         4,
    #         "Good-bye Senna",
    #         "I migrated all Senna system!",
    #         3
    #       ],
    #       [
    #         5,
    #         "Good-bye Tritonn",
    #         "I also migrated all Tritonn system!",
    #         3
    #       ]
    #     ]
    #   ]
    # ]

The `select` command searches records that `n_likes` column value is less than `11` from `Entries` table.

See Query syntax for other operations.

#### 8.4.17.4.2.3. `filter`

It specifies the filter text. Normally, it is used for complex search conditions. `filter` can be used with `query` parameter. If both `filter` and `query` are specified, there are conbined with logical and. It means that matched records should be matched against both `filter` and `query`.

`filter` parameter is designed for complex conditions. A filter text should be formated in Script syntax. The syntax is similar to ECMAScript. For example, `column == "value"` means that the value of `column` column is equal to `"value"`. `column < value` means that the value of `column` column is less than `value`.

Here is a simple equality operator search example.

Execution example:

    select Entries --filter '_key == "Groonga"'
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         1
    #       ],
    #       [
    #         [
    #           "_id",
    #           "UInt32"
    #         ],
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "content",
    #           "Text"
    #         ],
    #         [
    #           "n_likes",
    #           "UInt32"
    #         ]
    #       ],
    #       [
    #         2,
    #         "Groonga",
    #         "I started to use groonga. It's very fast!",
    #         10
    #       ]
    #     ]
    #   ]
    # ]

The `select` command searches records that `_key` column value is `Groonga` from `Entries` table.

Here is a simple less than operator search example.

Execution example:

    select Entries --filter 'n_likes < 11'
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         4
    #       ],
    #       [
    #         [
    #           "_id",
    #           "UInt32"
    #         ],
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "content",
    #           "Text"
    #         ],
    #         [
    #           "n_likes",
    #           "UInt32"
    #         ]
    #       ],
    #       [
    #         1,
    #         "The first post!",
    #         "Welcome! This is my first post!",
    #         5
    #       ],
    #       [
    #         2,
    #         "Groonga",
    #         "I started to use groonga. It's very fast!",
    #         10
    #       ],
    #       [
    #         4,
    #         "Good-bye Senna",
    #         "I migrated all Senna system!",
    #         3
    #       ],
    #       [
    #         5,
    #         "Good-bye Tritonn",
    #         "I also migrated all Tritonn system!",
    #         3
    #       ]
    #     ]
    #   ]
    # ]

The `select` command searches records that `n_likes` column value is less than `11` from `Entries` table.

See Script syntax for other operators.

### 8.4.17.4.3. Advanced search parameters

#### 8.4.17.4.3.1. `match_escalation_threshold`

It specifies threshold to determine whether search storategy escalation is used or not. The threshold is compared against the number of matched records. If the number of matched records is equal to or less than the threshold, the search storategy escalation is used. See 検索 about the search storategy escalation.

The default threshold is 0. It means that search storategy escalation is used only when no records are matched.

The default threshold can be customized by one of the followings.

> - `--with-match-escalation-threshold` option of configure
> - `--match-escalation-threshold` option of groogna command
> - `match-escalation-threshold` configuration item in configuration file

Here is a simple `match_escalation_threshold` usage example. The first `select` doesn't have `match_escalation_threshold` parameter. The second `select` has `match_escalation_threshold` parameter.

Execution example:

    select Entries --match_columns content --query groo
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         1
    #       ],
    #       [
    #         [
    #           "_id",
    #           "UInt32"
    #         ],
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "content",
    #           "Text"
    #         ],
    #         [
    #           "n_likes",
    #           "UInt32"
    #         ]
    #       ],
    #       [
    #         2,
    #         "Groonga",
    #         "I started to use groonga. It's very fast!",
    #         10
    #       ]
    #     ]
    #   ]
    # ]
    select Entries --match_columns content --query groo --match_escalation_threshold -1
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         0
    #       ],
    #       [
    #         [
    #           "_id",
    #           "UInt32"
    #         ],
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "content",
    #           "Text"
    #         ],
    #         [
    #           "n_likes",
    #           "UInt32"
    #         ]
    #       ]
    #     ]
    #   ]
    # ]

The first `select` command searches records that contain a word `groo` in `content` column value from `Entries` table. But no records are matched because the `TokenBigram` tokenizer tokenizes `groonga` to `groonga` not `gr|ro|oo|on|ng|ga`. (The `TokenBigramSplitSymbolAlpha` tokenizer tokenizes `groonga` to `gr|ro|oo|on|ng|ga`. See Tokenizers for details.) It means that `groonga` is indexed but `groo` isn't indexed. So no records are matched against `groo` by exact match. In the case, the search storategy escalation is used because the number of matched records (0) is equal to `match_escalation_threshold` (0). One record is matched against `groo` by unsplit search.

The second `select` command also searches records that contain a word `groo` in `content` column value from `Entries` table. And it also doesn't found matched records. In this case, the search storategy escalation is not used because the number of matched records (0) is larger than `match_escalation_threshold` (-1). So no more searches aren't executed. And no records are matched.

#### 8.4.17.4.3.2. `query_expansion`

It's for query expansion. Query expansion substitutes specific words to another words in query. Nomally, it's used for synonym search.

It specifies a column that is used to substitute `query` parameter value. The format of this parameter value is "`${TABLE}.${COLUMN}`". For example, "`Terms.synonym`" specifies `synonym` column in `Terms` table.

Table for query expansion is called "substitution table". Substitution table's key must be `ShortText`. So array table (`TABLE_NO_KEY`) can't be used for query expansion. Because array table doesn't have key.

Column for query expansion is called "substitution column". Substitution column's value type must be `ShortText`. Column type must be vector (`COLUMN_VECTOR`).

Query expansion substitutes key of substitution table in query with values in substitution column. If a word in `query` is a key of substitution table, the word is substituted with substitution column value that is associated with the key. Substition isn't performed recursively. It means that substitution target words in substituted query aren't substituted.

Here is a sample substitution table to show a simple `query_expansion` usage example.

Execution example:

    table_create Thesaurus TABLE_PAT_KEY|KEY_NORMALIZE ShortText
    # [[0, 1337566253.89858, 0.000355720520019531], true]
    column_create Thesaurus synonym COLUMN_VECTOR ShortText
    # [[0, 1337566253.89858, 0.000355720520019531], true]
    load --table Thesaurus
    [
    {"_key": "mroonga", "synonym": ["mroonga", "tritonn", "groonga mysql"]},
    {"_key": "groonga", "synonym": ["groonga", "senna"]}
    ]
    # [[0, 1337566253.89858, 0.000355720520019531], 2]

`Thesaurus` substitution table has two synonyms, `"mroonga"` and `"groonga"`. If an user searches with `"mroonga"`, groonga searches with `"((mroonga) OR (tritonn) OR (groonga mysql))"`. If an user searches with `"groonga"`, groonga searchs with `"((groonga) OR (senna))"`. Nomrally, it's good idea that substitution table has `KEY_NORMALIZE` flag. If the flag is used, substitute target word is matched in case insensitive manner.

Note that those synonym values include the key value such as `"mroonga"` and `"groonga"`. It's recommended that you include the key value. If you don't include key value, substituted value doesn't include the original substitute target value. Normally, including the original value is better search result. If you have a word that you don't want to be searched, you should not include the original word. For example, you can implement "stop words" by an empty vector value.

Here is a simple `query_expansion` usage example.

Execution example:

    select Entries --match_columns content --query "mroonga"
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         1
    #       ],
    #       [
    #         [
    #           "_id",
    #           "UInt32"
    #         ],
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "content",
    #           "Text"
    #         ],
    #         [
    #           "n_likes",
    #           "UInt32"
    #         ]
    #       ],
    #       [
    #         3,
    #         "Mroonga",
    #         "I also started to use mroonga. It's also very fast! Really fast!",
    #         15
    #       ]
    #     ]
    #   ]
    # ]
    select Entries --match_columns content --query "mroonga" --query_expansion Thesaurus.synonym
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         2
    #       ],
    #       [
    #         [
    #           "_id",
    #           "UInt32"
    #         ],
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "content",
    #           "Text"
    #         ],
    #         [
    #           "n_likes",
    #           "UInt32"
    #         ]
    #       ],
    #       [
    #         3,
    #         "Mroonga",
    #         "I also started to use mroonga. It's also very fast! Really fast!",
    #         15
    #       ],
    #       [
    #         5,
    #         "Good-bye Tritonn",
    #         "I also migrated all Tritonn system!",
    #         3
    #       ]
    #     ]
    #   ]
    # ]
    select Entries --match_columns content --query "((mroonga) OR (tritonn) OR (groonga mysql))"
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         2
    #       ],
    #       [
    #         [
    #           "_id",
    #           "UInt32"
    #         ],
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "content",
    #           "Text"
    #         ],
    #         [
    #           "n_likes",
    #           "UInt32"
    #         ]
    #       ],
    #       [
    #         3,
    #         "Mroonga",
    #         "I also started to use mroonga. It's also very fast! Really fast!",
    #         15
    #       ],
    #       [
    #         5,
    #         "Good-bye Tritonn",
    #         "I also migrated all Tritonn system!",
    #         3
    #       ]
    #     ]
    #   ]
    # ]

The first `select` command doesn't use query expansion. So a record that has `"tritonn"` isn't found. The second `select` command uses query expansion. So a record that has `"tritonn"` is found. The third `select` command doesn't use query expansion but it is same as the second `select` command. The third one uses expanded query.

Each substitute value can contain any Query syntax syntax such as `(...)` and `OR`. You can use complex substitution by using those syntax.

Here is a complex substitution usage example that uses query syntax.

Execution example:

    load --table Thesaurus
    [
    {"_key": "popular", "synonym": ["popular", "n_likes:>=10"]}
    ]
    # [[0, 1337566253.89858, 0.000355720520019531], 1]
    select Entries --match_columns content --query "popular" --query_expansion Thesaurus.synonym
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         2
    #       ],
    #       [
    #         [
    #           "_id",
    #           "UInt32"
    #         ],
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "content",
    #           "Text"
    #         ],
    #         [
    #           "n_likes",
    #           "UInt32"
    #         ]
    #       ],
    #       [
    #         2,
    #         "Groonga",
    #         "I started to use groonga. It's very fast!",
    #         10
    #       ],
    #       [
    #         3,
    #         "Mroonga",
    #         "I also started to use mroonga. It's also very fast! Really fast!",
    #         15
    #       ]
    #     ]
    #   ]
    # ]

The `load` command registers a new synonym `"popular"`. It is substituted with `((popular) OR (n_likes:>=10))`. The substituted query means that "popular" is containing the word "popular" or 10 or more liked entries.

The `select` command outputs records that `n_likes` column value is equal to or more than `10` from `Entries` table.

#### 8.4.17.4.3.3. `query_flags`

It customs `query` parameter syntax. You cannot update column value by `query` parameter by default. But if you specify `ALLOW_COLUMN|ALLOW_UPDATE` as `query_flags`, you can update column value `query`.

Here are available values:

- `ALLOW_PRAGMA`
- `ALLOW_COLUMN`
- `ALLOW_UPDATE`
- `NONE`

`ALLOW_PRAGMA` enables pragma at the head of `query`.

`ALLOW_COLUMN` enables search againt columns that are not included in `match_columns`. To specify column, there are `COLUMN:...` syntaxes.

`ALLOW_UPDATE` enables column update by `query` with `COLUMN:=NEW_VALUE` syntax. `ALLOW_COLUMN` is also required to update column because the column update syntax specifies column.

`NONE` is just ignores. You can use `NONE` for specifying no flags.

They can be combined by separated `|` such as `ALLOW_COLUMN|ALLOW_UPDATE`.

The default value is `ALLOW_PRAGMA|ALLOW_COLUMN`.

TODO: example

See also Query syntax.

### 8.4.17.4.4. Output related parameters

#### 8.4.17.4.4.1. `output_columns`

It specifies output columns separated by `,`.

Here is a simple `output_columns` usage example.

Execution example:

    select Entries --output_columns '_id, _key' --limit 1
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         5
    #       ],
    #       [
    #         [
    #           "_id",
    #           "UInt32"
    #         ],
    #         [
    #           "_key",
    #           "ShortText"
    #         ]
    #       ],
    #       [
    #         1,
    #         "The first post!"
    #       ]
    #     ]
    #   ]
    # ]

The `select` command just outputs `_id` and `_key` column values.

`*` is a special value. It means that all columns that are not 疑似カラム (pseudo_column).

Here is a `*` usage example.

Execution example:

    select Entries --output_columns '_key, *' --limit 1
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         5
    #       ],
    #       [
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "content",
    #           "Text"
    #         ],
    #         [
    #           "n_likes",
    #           "UInt32"
    #         ]
    #       ],
    #       [
    #         "The first post!",
    #         "Welcome! This is my first post!",
    #         5
    #       ]
    #     ]
    #   ]
    # ]

The `select` command outputs `_key` pseudo column, `content` column and `n_likes` column values but doesn't output `_id` pseudo column value.

The default value is `_id, _key, *`. It means that all column values except `_score` are outputted.

#### 8.4.17.4.4.2. `sortby`

It specifies sort keys separated by `,`. Each sort key is column name.

Here is a simple `sortby` usage example.

Execution example:

    select Entries --sortby 'n_likes, _id'
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         5
    #       ],
    #       [
    #         [
    #           "_id",
    #           "UInt32"
    #         ],
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "content",
    #           "Text"
    #         ],
    #         [
    #           "n_likes",
    #           "UInt32"
    #         ]
    #       ],
    #       [
    #         4,
    #         "Good-bye Senna",
    #         "I migrated all Senna system!",
    #         3
    #       ],
    #       [
    #         5,
    #         "Good-bye Tritonn",
    #         "I also migrated all Tritonn system!",
    #         3
    #       ],
    #       [
    #         1,
    #         "The first post!",
    #         "Welcome! This is my first post!",
    #         5
    #       ],
    #       [
    #         2,
    #         "Groonga",
    #         "I started to use groonga. It's very fast!",
    #         10
    #       ],
    #       [
    #         3,
    #         "Mroonga",
    #         "I also started to use mroonga. It's also very fast! Really fast!",
    #         15
    #       ]
    #     ]
    #   ]
    # ]

The `select` command sorts by `n_likes` column value in ascending order. For records that has the same `n_likes` are sorted by `_id` in ascending order. `"Good-bye Senna"` and `"Good-bye Tritonn"` are the case.

If you want to sort in descending order, add `-` before column name.

Here is a descending order `sortby` usage example.

Execution example:

    select Entries --sortby '-n_likes, _id'
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         5
    #       ],
    #       [
    #         [
    #           "_id",
    #           "UInt32"
    #         ],
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "content",
    #           "Text"
    #         ],
    #         [
    #           "n_likes",
    #           "UInt32"
    #         ]
    #       ],
    #       [
    #         3,
    #         "Mroonga",
    #         "I also started to use mroonga. It's also very fast! Really fast!",
    #         15
    #       ],
    #       [
    #         2,
    #         "Groonga",
    #         "I started to use groonga. It's very fast!",
    #         10
    #       ],
    #       [
    #         1,
    #         "The first post!",
    #         "Welcome! This is my first post!",
    #         5
    #       ],
    #       [
    #         4,
    #         "Good-bye Senna",
    #         "I migrated all Senna system!",
    #         3
    #       ],
    #       [
    #         5,
    #         "Good-bye Tritonn",
    #         "I also migrated all Tritonn system!",
    #         3
    #       ]
    #     ]
    #   ]
    # ]

The `select` command sorts by `n_likes` column value in descending order. But ascending order is used for sorting by `_id`.

You can use `_score` pseudo column in `sortby` if you use `query` or `filter` parameter.

Execution example:

    select Entries --match_columns content --query fast --sortby -_score --output_columns '_key, _score'
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         2
    #       ],
    #       [
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "_score",
    #           "Int32"
    #         ]
    #       ],
    #       [
    #         "Mroonga",
    #         2
    #       ],
    #       [
    #         "Groonga",
    #         1
    #       ]
    #     ]
    #   ]
    # ]

The `select` command sorts matched records by hit score in descending order and outputs record key and hit score.

If you use `_score` without `query` nor `filter` parameters, it's just ignored but get a warning in log file.

#### 8.4.17.4.4.3. `offset`

It specifies offset to determine output records range. Offset is zero-origin. `--offset 1` means output range is started from the 2nd record.

Execution example:

    select Entries --sortby _id --offset 3 --output_columns _key
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         5
    #       ],
    #       [
    #         [
    #           "_key",
    #           "ShortText"
    #         ]
    #       ],
    #       [
    #         "Good-bye Senna"
    #       ],
    #       [
    #         "Good-bye Tritonn"
    #       ]
    #     ]
    #   ]
    # ]

The `select` command outputs from the 4th record.

You can specify negative value. It means that `the number of matched records + offset`. If you have 3 matched records and specify `--offset -2`, you get records from the 1st (`3 + -2 = 1`) record to the 3rd record.

Execution example:

    select Entries --sortby _id --offset -2 --output_columns _key
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         5
    #       ],
    #       [
    #         [
    #           "_key",
    #           "ShortText"
    #         ]
    #       ],
    #       [
    #         "Good-bye Senna"
    #       ],
    #       [
    #         "Good-bye Tritonn"
    #       ]
    #     ]
    #   ]
    # ]

The `select` command outputs from the 4th record because the total number of records is `5`.

The default value is `0`.

#### 8.4.17.4.4.4. `limit`

It specifies the max number of output records. If the number of matched records is less than `limit`, all records are outputted.

Here is a simple `limit` usage example.

Execution example:

    select Entries --sortby _id --offset 2 --limit 3 --output_columns _key
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         5
    #       ],
    #       [
    #         [
    #           "_key",
    #           "ShortText"
    #         ]
    #       ],
    #       [
    #         "Mroonga"
    #       ],
    #       [
    #         "Good-bye Senna"
    #       ],
    #       [
    #         "Good-bye Tritonn"
    #       ]
    #     ]
    #   ]
    # ]

The `select` command outputs the 3rd, the 4th and the 5th records.

You can specify negative value. It means that `the number of matched records + limit + 1`. For example, `--limit -1` outputs all records. It's very useful value to show all records.

Here is a simple negative `limit` value usage example.

Execution example:

    select Entries --limit -1
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         5
    #       ],
    #       [
    #         [
    #           "_id",
    #           "UInt32"
    #         ],
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "content",
    #           "Text"
    #         ],
    #         [
    #           "n_likes",
    #           "UInt32"
    #         ]
    #       ],
    #       [
    #         1,
    #         "The first post!",
    #         "Welcome! This is my first post!",
    #         5
    #       ],
    #       [
    #         2,
    #         "Groonga",
    #         "I started to use groonga. It's very fast!",
    #         10
    #       ],
    #       [
    #         3,
    #         "Mroonga",
    #         "I also started to use mroonga. It's also very fast! Really fast!",
    #         15
    #       ],
    #       [
    #         4,
    #         "Good-bye Senna",
    #         "I migrated all Senna system!",
    #         3
    #       ],
    #       [
    #         5,
    #         "Good-bye Tritonn",
    #         "I also migrated all Tritonn system!",
    #         3
    #       ]
    #     ]
    #   ]
    # ]

The `select` command outputs all records.

The default value is `10`.

#### 8.4.17.4.4.5. `scorer`

TODO: write in English and add example.

検索条件にマッチする全てのレコードに対して適用するgrn_exprをscript形式で指定します。

scorerは、検索処理が完了し、ソート処理が実行される前に呼び出されます。従って、各レコードのスコアを操作する式を指定しておけば、検索結果のソート順序をカスタマイズできるようになります。

### 8.4.17.4.5. Facet related parameters

#### 8.4.17.4.5.1. `drilldown`

TODO: write in English and add example.

グループ化のキーとなるカラム名のリストをカンマ(',')区切りで指定します。検索条件にマッチした各レコードを出力したのちに、drilldownに指定されたカラムの値が同一であるレコードをとりまとめて、それぞれについて結果レコードを出力します。

#### 8.4.17.4.5.2. `drilldown_sortby`

TODO: write in English and add example.

drilldown条件に指定されたカラムの値毎にとりまとめられたレコードについて、ソートキーとなるカラム名のリストをカンマ(',')区切りで指定します。sortbyパラメータと同様に昇降順を指定できます。

#### 8.4.17.4.5.3. `drilldown_output_columns`

TODO: write in English and add example.

drilldown条件に指定されたカラムの値毎にとりまとめられたレコードについて、出力するカラム名のリストをカンマ(',')区切りで指定します。

#### 8.4.17.4.5.4. `drilldown_offset`

TODO: write in English and add example.

drilldown条件に指定されたカラムの値毎にとりまとめられたレコードについて、出力対象となる最初のレコードの番号を0ベースで指定します。デフォルト値は0です。drilldown_offsetに負の値を指定した場合は、ヒットした件数 + drilldown_offsetによって算出される値が指定されたものとみなされます。

#### 8.4.17.4.5.5. `drilldown_limit`

TODO: write in English and add example.

drilldown条件に指定されたカラムの値毎にとりまとめられたレコードについて、出力を行うレコードの件数を指定します。デフォルト値は10です。実際には、drilldown_offset + drilldown_limit がヒットした件数を超えない範囲でレコードが出力されます。drilldown_limitに負の値を指定した場合は、ヒットした件数 + drilldown_limit + 1 によって算出される値が指定されたものとみなされます。

### 8.4.17.4.6. Cache related parameter

#### 8.4.17.4.6.1. `cache`

TODO: write in English and add example.

クエリキャッシュに関する動作を設定します。

`no`

> 検索結果をクエリキャッシュに残しません。キャッシュして再利用される可能性が低いクエリに対して用います。キャッシュ容量は有限です。有効なキャッシュが多くヒットするために、このパラメータは有効です。

## 8.4.17.5. 返値

TODO: write in English and add example.

以下のようなjson形式で値が返却されます。

    [[リターンコード, 処理開始時間, 処理時間], [検索結果, ドリルダウン結果]]

`リターンコード`

> grn_rcに対応する数値が返されます。0(GRN_SUCCESS)以外の場合は、続いてエラー内容を示す 文字列が返されます。

`処理開始時間`

> 処理を開始した時間について、1970年1月1日0時0分0秒を起点とした秒数を小数で返します。

`処理時間`

> 処理にかかった秒数を返します。

`検索結果`

> drilldown条件が実行される前の検索結果が以下のように出力されます。:

>     [[検索件数], [[カラム名1,カラム型1],..], 検索結果1,..]

> `検索件数`

> > 検索件数が出力されます。

> `カラム名n`

> > output_columnsに指定された条件に従って、対象となるカラム名が出力されます。

> `カラム型n`

> > output_columnsに指定された条件に従って、対象となるカラム型が出力されます。

> `検索結果n`

> > output_columns, offset, limitによって指定された条件に従って各レコードの値が出力されます。

`drilldown結果`

> drilldown処理の結果が以下のように出力されます。:

>     [[[件数], [[カラム名1,カラム型1],..], 検索結果1,..],..]

> `件数`

> > drilldownに指定されたカラムの値の異なり数が出力されます。

> `カラム名n`

> > drilldown_output_columnsに指定された条件に従って、対象となるカラム名が出力されます。

> `カラム型n`

> > drilldown_output_columnsに指定された条件に従って、対象となるカラム型が出力されます。

> `ドリルダウン結果n`

> > drilldown_output_columns, drilldown_offset, drilldown_limitによって指定された条件に従って各レコードの値が出力されます。

## 8.4.17.6. See also

> - Query syntax
> - Script syntax

### Navigation

- index
- next |
- previous |
- groonga v2.0.5-154-g8f85693 documentation »
- 8. リファレンスマニュアル »
- 8.4. コマンド »