# groonga - An open-source fulltext search engine and column store.

- 日本語

# News - 4.x

## Release 4.1.1 - 2015-01-29

### Improvements

- [windows] Improved to show error message instead of error code. It's easy to understand.
- [experimental][windows] Supported small start file size for database file. In previous versions, Groonga consumed larger initial file size on windows in contrast to UNIX like environment. To enable this feature, use `GRN_IO_VERSION=1` environment variable. This feature isn't enabled by default because it is in experimental stage.
- Supported `SUM`, `MIN`, `MAX`, `AVG` feature for drilldown. This feature is supported by extending newer drilldown syntax which supports label. Specify `SUM`, `MIN`, `MAX` or `AVG` for drilldown_calc_types and target column for drilldown_calc_target. The return value is calculated based on drilldown result in group.
- [experimental] Added `--with-jemalloc` option to use `jemalloc` for `malloc()/free()`. It'll improve performance on multi-threading usage. For example, the performance of groonga executable file or Mroonga will be improved. Note that this feature doesn't affect to groonga-httpd because groonga-httpd uses multi-process model instead of multi-threading.
- groonga-httpd: Supported long error message over 2048 bytes. By this change, long error message (>2048) is not truncated.
- [dump] Supported streaming output.

### Fixes

- Fixed mismatch between error code and error message when `strerror()` is failed. There was a case that Groonga shows incorrect error code as syscall error.
- Fixed to show errno in error message when unknow error is occurred.
- [windows] Fixed to return proper error code.
- Fixed a bug that there is only one section for multi-column index for match_columns, it doesn't return correct search results. For example, consider the case that there is multi-column index which is named as `memo_index` for `title` column and `content` column. Thus, this bug occurs when `memo_index[0]` which means index for `title` column is specified for match_columns.
- mecab: Added workaround for `mecab_strerror(NULL)` on MeCab 0.993. Without this workaround, Groonga aborts if there is no MeCab dictionary. [Reported by Shuhei Tanuma]
- groonga-httpd: Fixed a bug that custom log file is ignored.
- groonga-httpd: Fixed a bug that empty dump doesn't return.

### Thanks

- Shuhei Tanuma

## Release 4.1.0 - 2015-01-09

It's a bug fix release of 4.0.9.

If you have already upgraded to 4.0.9, please upgrade to 4.1.0. And re-create your all indexes. You can re-create your indexes by removing and re-creating your indexes such as:

    column_remove TABLE INDEX_COLUMN
    column_create TABLE INDEX_COLUMN COLUMN_INDEX|...

See also

- column_remove
- column_create

Note

If you didn't change your indexes by Groonga 4.0.9, you don't need to re-create your indexes. "change" means "add", "remove" and "update". If you use your indexes only for "search" by Groonga 4.0.9, you don't need to re-create your indexes.

### Fixes

- Fixed a bug that index may be broken. But the following bug fixed in Groonga 4.0.9 is re-appeared.
  - Fixed a crash bug that may be occurred by deleting or updating many indexed values at once. [groonga-dev,02942] [Reported by kashihara]

## Release 4.0.9 - 2014-12-29

Caution

Use 4.1.0 instead of 4.0.9. 4.0.9 has a serious bug that may break index.

### Improvements

- [groonga-httpd] Marked as stable.
- [doc] Described about `groonga-token-filter-stem` package. [GitHub#259] [Patch by Masafumi Yokoyama]
- [build] Supported `make install` without `make`. [Reported by akira yamada]
- [experimental] Added request_cancel command that cancels running request.
- [experimental] Added `grn_db_recover()`.
- Supported indexing vector of fixed size data such as vector of `UInt64`. [groonga-dev,03014] [Reported by funa 1g]
- [experimental] Added grndb.
- [deb] Supported armhf architecture.
- [Groonga HTTP server] Supported HEAD.
- [doc] Updated Introduction. [GitHub#270] [Patch by Yuya TAMANO]
- [doc] Updated Others. [GitHub#271] [Patch by Hiroshi Hatake]
- [TokenFilterStem] Supported token that consists with only upper-case alphabets, digits, `-` and `'`. It means that TokenFilterStem can work with MySQL compatible normalizers.
- [truncate] Supported column.
- [truncate] Changed parameter name to `target_name` from `table`. `table` parameter can be used for backward compatibility but it shouldn't used for newly written code.
- Exported `grn_column_truncate()`.
- Added lock_clear.
- Marked clearlock deprecated.
- [backward incompatible] Stopped to set world writable permission for newly created files on `umask(0)` environment such as daemoned groonga-httpd. It's a backward incompatible change but done because the previous behavior has a security risk.

### Fixes

- Fixed build error on BSD based system. [GitHub#253] [Reported by Hiroshi Hatake]
- Fixed build error on OS X. [GitHub#253] [Reported by Hiroshi Hatake]
- Fixed a regression bug that equal operation against nonexistent reference always matches.
- Fixed a crash bug that may be occurred by deleting or updating many indexed values at once. [groonga-dev,02942] [Reported by kashihara]
- Fixed a regression bug that static index construction doesn't work on BSD based system. [GitHub#264] [Reported by Ayumu Osanai]
- Fixed a bug that `--with-libstemmer` configure option is ignored.
- [deb] Fixed a bug that logrotate script has a syntax error.
- Fixed a bug that grouped result can't be got when Command version is `2`. [GitHub ranguba/rroonga#38] [Reported by YUKI "Piro" Hiroshi]

### Thanks

- Hiroshi Hatake
- Masafumi Yokoyama
- akira yamada
- kashihara
- Ayumu Osanai
- funa 1g
- YUKI "Piro" Hiroshi
- Yuya TAMANO

## Release 4.0.8 - 2014-11-29

### Improvements

- [deb] Supported Ubuntu 12.04 again because Travis-CI runs on it.
- [mruby] Supported pretty print - `"pp"` for debugging. And supported to show internal objects more descriptive format when inspecting it.
- Supported full text search query without index if one column is specified for `--match_columns` even though sequential search is executed by `--query` and `--match_columns` combination. [groonga-dev,02902] [Reported by Atsushi Shinoda]
- [experimental] in_values supports auto sequential search mode. There is a case that sequential search is faster than index search when the number of narrowed down records is small enough in contrast to the number of expected records to narrow down by `in_values` with `AND` operation which use indexes. The value of `GRN_IN_VALUES_TOO_MANY_INDEX_MATCH_RATIO` is used as threshold whether Groonga execute sequential search or search with indexes in such a case.
- [mruby] Supported to load relative path of script.
- Supported test suite to write query optimizer test more easily with mruby enabled build.
- Supported offline index construction for token filters. [GitHub#239] [Patch by Naoya Murakami]
- [experimental] between supports auto sequential search mode. Use the value of `GRN_BETWEEN_TOO_MANY_INDEX_MATCH_RATIO` as threshold. Basic concept is similar to `GRN_IN_VALUES_TOO_MANY_INDEX_MATCH_RATIO`.
- [experimental] Supported multiple drilldowns for select command. The syntax is extended to support this feature. The new one is `--drilldown[LABEL1].keys COLUMN1,COLUMN2` for keys, `--drilldown[LABEL1].output_columns _value.COLUMN1,_value.COLUMN2,_nsubrecs` for output columns. In the previous versions, a client application must implement by oneself for drilldown by multiple keys. Use `_key[N]` for `--drilldown[LABEL1].output_columns` to show original value of key in records.
- Supported to search `'reference_column @ "query"'` without index.
- [experimental] Added range_filter command. It is faster than select command to fetch specific range of records in most cases, because it doesn't returns the number of matched records. There are limitations which doesn't support drilldown and so on. Thus, `range_filter` is suitable for extracting first `N` records in specific range for ranking or something else.
- [httpd] Supported `groonga-httpd` on windows.
- Supported vector column for drilldown.
- [deb] Changed `groonga-httpd` as default HTTP server package. `groonga-server-http` is deprecated. Note that Groonga HTTP server functionality itself is still exists in Groonga. If you just want to use simple HTTP server in traditional way, specify proper arguments by command line for groonga executable file.
- Added configuration file for Windows CI on AppVeyor. [groonga-dev,02970,02981] [Patch by Hiroshi Hatake]

### Fixes

- [doc] Fixed duplicated execution example path about table_create. [GitHub groonga.org#14] [Reported by Masafumi Yokoyama]
- Fixed a bug that nested indexed full text search against `X._key` returns empty results.
- Fixed flags parameter which is strict in BSD `mkostemp(3)`. [GitHub#240] [Patch by Jun Kuriyama]
- Fixed a bug that select command reports invalid value type (table) as index column value type. It was fixed to `UInt32`. Note that it affects only the value of response header. The search results is not changed at all.
- Fixed a bug that compare operations such as `<`, `<=`, `>`, `>=`, `!=` with Time and String type returns invalid results.
- Fixed a bug that prefix search can't be executed against single character such as `"x*"`. [GitHub#242] [Patch by Naoya Murakami]
- Fixed a memory leak on drilldown `ShortText` value with `--filter`.
- Fixed missing `","` for `"DDL:set_token_filters"` log. [Reported by Naoya Murakami]

### Thanks

- Atsushi Shinoda
- Masafumi Yokoyama
- Naoya Murakami
- Jun Kuriyama
- Hiroshi Hatake

## Release 4.0.7 - 2014-10-29

### Improvements

- [windows] Supported Groonga with mruby on Windows.
- Added token filter plugin API.
- Added TokenFilterStopWord token filter plugin. It treats records of `is_stop_word` column as actual stop words.
- [doc] Added Client section which introduces link to client libraries. [GitHub#210] [Patch by Naoya Murakami]
- [doc] Reconstructed documentation. Suggest and Geolocation search are grouped under Reference manual. [GitHub#206] [Patch by Naoya Murakami]
- [deb] Added stop word token filter plugin. [GitHub#216] [Patch by Hiroshi Hatake]
- [deb] Clarified licenses about used/bundled files in debian/copyright.
- Dropped LZO compression support.
- Added LZ4 compression support. [GitHub#223] [Patch by Naoya Murakami]
- Added in_values. We recommends to use it instead of `==` and `OR` combinations ( `((COLUMN == "a") OR (COLUMN == "b") OR (COLUMN == "c"))` ) because of better performance.
- [load] Supported to update existing record in `TABLE_NO_KEY` by specifying a record by `_id`.
- Added TokenFilterStem token filter plugin. It provides stemming feature. To use this token filter, you need to install libstemmer library.
- [doc] Added Token filters documentation. [GitHub#227] [Patch by Naoya Murakami]

### Fixes

- [doc] Fixed link of the Ranguba project. [GitHub#212] [Patch by Naoya Murakami]
- [httpd] Updated bundled nginx version to 1.7.6. [GitHub#208] [Patch by Hiroshi Hatake]
- [power8] Fixed build failure. See http://lists.askmonty.org/pipermail/commits/2014-October/006752.html [Patch by Sergey Vojtovich]
- Fixed a bug that URL encoding fails if `char` type is build as `unsigned char` type. [GitHub#217]
- Fixed a bug that a `TABLE_DAT_KEY` table returns empty results for select. [GitHub#220]
- Fixed a memory leak about inspecting vector column. It occurs when something error reports about vector column.
- Fixed memory leaks in zlib/LZO compressed column [GitHub#221] [Patch by Naoya Murakami]
- Fixed a bug that the value of `Int8` column which is less than 100 may return true in `--filter`. By this issue, not intended records may be included as search results.
- Fixed to handle binary operation for vector column as an error.
- Fixed a bug that `!XXX` may return inverted result on big endian environment.
- Fixed a memory leak on converting weight vector to text.
- [dump] Fixed token filters and `TABLE_NO_KEY` related error in dump. [GitHub#215] [Patch by Naoya Murakami]

### Thanks

- Naoya Murakami
- Hiroshi Hatake
- Sergey Vojtovich

## Release 4.0.6 - 2014-09-29

### Improvements

- [deb] Updated to follow the rule about machine-readable debian/copyright file. [Reported by Hideki Yamane]
- [deb][rpm] Changed to use signal instead of HTTP to control nginx. This way meets nginx style init script.
- [doc] Updated English documentation about [ Executables ]. [GitHub#196] [Patch by Masafumi Yokoyama]
- [OS X] Added missing sphinx error message for Homebrew users. [GitHub#198] [Patch by cosmo0920]
- [mruby] Query optimizer by mruby became a practical for some range select query. For example, mruby optimizer overhead is negligible enough for query such as "between(X, 0, 'exclude', 100, 'include')" - "X > 0 && X <= 100".
- [doc] Changed to recommend GitHub for reporting issues. Until now, Redmine account must be approved by administrator, so you can not report at feel free. [GitHub#202] [Patch by cosmo0920]
- [doc] Translated C API document. [GitHub#203] [Patch by NOKUBI Takatsugu]
- Supported to customize similarity threshold by `'*S10"..."'` syntax.
- Added source column validation to column_create. If source column doesn't exist, column_create returns `false` and column isn't created.
- [debian] Dropped Debian jessie and sid support.

### Fixes

- Fixed a bug that similar search doesn't use IDF (Inverse Document Frequency) for choosing target tokens. In the previous versions, it used token ID instead of IDF. [groonga-dev,02731] [groonga-dev,02754] [Reported by Satoshi Mitani, warp kawada]
- Fixed a crash bug when deleting a record. If the table of the record has `_key` and `_key` is indexed from two or more index columns, deleting causes a crash.
- Fixed compiler warning about unused variable 'e'. [GitHub#197] [Patch by cosmo0920]
- [fedora][httpd] Fixed service start failure without log directory.
- [mruby] Fixed build error with `--disable-mruby`. [GitHub#200] [Patch by cosmo0920]
- Fixed build warning about CMP0014 for MariaDB bundled Mroonga package. [GitHub#201] [Patch by cosmo0920]
- Fixed a bug that "*" is parsed as prefix search with empty string. [groonga-dev,02796] [Reported by Kazuhiko]
- [httpd] Updated bundled nginx version to the latest mainline (1.7.5). This update contains vulnerability fix (CVE-2014-3616).

### Thanks

- Hideki Yamane
- Masafumi Yokoyama
- cosmo0920
- NOKUBI Takatsugu
- Satoshi Mitani
- warp kawada
- Kazuhiko

## Release 4.0.5 - 2014-08-29

### Improvements

- Dropped Ubuntu 13.10 (Saucy Salamander).
- Supported to show weight vector element as XML.
- [ between ] Supported outside of key range. [GitHub#183] [Patch by Naoya Murakami]
- [mruby] Changed to use Onigmo as regexp implementation.
- Added highlight_full and highlight_html which tags output text. [GitHub#185] [Patch by Naoya Murakami]
- [doc] Updated outdated functions/commands reference [GitHub#188] [GitHub#189] [Patch by Naoya Murakami]
- [benchmark] Added average and median in benchmark result.
- Exported `grn_selector_func()` and `grn_proc_set_selector()` for plugin developer. [GitHub#190] [Patch by Naoya Murakami]
- [ tokenize ] Added "mode" option to control behavior. You can use `GET` or `ADD` as a mode. [GitHub#191] [GitHub#192] [Patch by Naoya Murakami]
- Supported near search by '*N"A Z"' syntax.
- Supported to customize behavior of tokenizer whether it skips token or skips token with position. This feature is useful for developing tokenizer plugin. [GitHub#193] [Patch by Naoya Murakami]
- [doc] Updated description that a bug report is accepted in Japanese on GitHub issue tracker. [Reported by 6elz] [groonga-dev,02651]
- [doc] Updated tutorial in English. [GitHub#194] [Patch by cosmo0920]

### Fixes

- [php] Fixed memory leak when `grn_ctx_recv()` is used. [GitHub#184] [Patch by @do-aki]
- [php] Fixed to avoid double free when `grn_ctx_close()` is called. [GitHub#184] [Patch by @do-aki]
- Fixed to support out-of-tree build again.
- [httpd] Updated bundled nginx version to the latest mainline (1.7.4). This update contains vulnerability fix.
- Fixed a bug that infinite loop occurs sometimes when updating indexes. [groonga-dev,02673] [Reported by MINAGAWA Sho]
- Fixed a bug that document frequency is invalid when Offline index construction is used. [groonga-dev,02691] [Reported by Naoya Murakami]

### Thanks

- @do-aki
- Naoya Murakami
- 6elz
- MINAGAWA Sho
- cosmo0920

## Release 4.0.4 - 2014-07-29

### Improvements

- [httpd] Updated bundled nginx version to the latest mainline (1.7.3).
- Supported weight in vector column.
- [ groonga-benchmark ] Added missing description about HTTP.
- [ Tutorial ] Updated English tutorial documentation.
- [example] Supported Ruby 1.9 or later for edict2grn.rb.
- [http][admin] Updated to use English as default language. [GitHub#182]

### Fixes

- Fixed to show right syntax error location in response.
- [httpd] Fixed build error on Clang. [GitHub clear-code/rurema-search#11] [Reported by niku]

### Thanks

- niku

## Release 4.0.3 - 2014-06-29

### Improvements

- [experimental][ Groonga HTTP server ] Supported POST.
- [windows] Bundled libwinpthread-1.dll. Because newer libgcc_s_sjlj-1.dll depends on libwinpthread-1.dll. [groonga-dev,02398] [Reported by Masafumi Yokoyama]
- [ groonga executable file ] Changed the default port number of GQTP to 10043. Because server packages use 10043 by default. [GitHub#172] [Patch by Naoya Murakami]
- [ normalize ] Added more validations for invalid argument. [groonga-dev,02409][GitHub:#174] [Reported by GMO Media, Inc.][Patch by Naoya Murakami]
- [ tokenize ] Added more validations for invalid argument.
- [ Groonga HTTP server ] Supported `Expect: 100-Continue` request.
- Exported `grn_proc_get_type()`.
- [ groonga-suggest-learner ] Added `--log-path` and `--log-level` options.
- Deprecated `GRN_CTX_USE_QL`.
- Deprecated `GRN_CTX_BATCH_MODE`.
- Added `grn_text_printf()`.
- Added `grn_text_vprintf()`.
- Removed limitation of one query log size.
- Added `grn_plugin_expr_var_init()`. [GitHub#175][Patch by Naoya Murakami]
- Added `grn_plugin_command_create()`. [GitHub#175][Patch by Naoya Murakami]
- [GitHub#176] Supported reference vector column with weight.
- [plugin] Used public API instead of internal API. [GitHub#177][GitHub#178] [Patch by Naoya Murakami]
- [doc][plugin] Added plugin APIs. [GitHub#179] [Patch by Naoya Murakami]
- [windows] Re-supported Visual C++ 2010. [groonga-dev,02454] [Reported by cosmo0920].

### Fixes

- [rpm][groonga-server-gqtp] Fixed a bug that HTTP protocol is used not GQTP. [GitHub#173] [Patch by Naoya Murakami]
- [ select ] Fixed a crash bug when adjuster has a syntax error.

### Thanks

- Masafumi Yokoyama
- Naoya Murakami
- GMO Media, Inc.
- cosmo0920

## Release 4.0.2 - 2014-05-29

### Improvements

- [doc] Updated documentation about groonga-suggest-learner.
- [doc] Added documentation about how to update files. [GitHub#160] [Patch by cosmo0920]
- [doc] Updated to caplitalized "Groonga" terms in documentation. [GitHub#162][GitHub#163][GitHub#164] [Patch by cosmo0920]
- Supported Ubuntu 14.04 (Trusty Tahr).
- Dropped Ubuntu 12.10.
- Migrated Ubuntu package distribution site to PPA on Launchpad. See Ubuntu for details.
- Handled all requests that start with `/d/` as API requests. You need to put files to directories that don't start with `/d/` to serve by Groonga HTTP service.
- [munin] Supported groonga-httpd. [Reported by Naoya Murakami]
- Supported daylight saving time. [#2546]
- [doc] Added a description about `--with-mecab` in Mac OS X.
- [http] Changed HTTP return code to `400 Bad Request` from `500 Internal Server Error` for syntax error case.
- [http][admin] Removed jQuery JSON plugin. [GitHub#168] [Patch by Tetsuharu OHZEKI]
- [http][admin] Enabled strict mode. [GitHub#169] [Patch by Tetsuharu OHZEKI]
- Exported getting variable APIs to Plugin. [GitHub#170] [Patch by Naoya Murakami]
  - Added `grn_plugin_proc_get_var()`.
  - Added `grn_plugin_proc_get_var_by_offset()`.
- [experimental] Added tokenizer_list. [GitHub#171] [Patch by Naoya Murakami]
- [experimental] Added normalizer_list. [GitHub#171] [Patch by Naoya Murakami]

### Fixes

- [index] Fixed a bug that wrong max segment. It causes a crash when you use all allocated resource for an index column. [#2438] [Reported by GMO Media, Inc.]
- [doc] Fixed a typo in CentOS. [GitHub#166] [Patch by Naoya Murakami]
- [doc] Fixed the wrong default value of `drilldown_output_columns` in select. [GitHub#167] [Patch by Naoya Murakami]
- [doc] Added a missing `\` escape exception in Query syntax. [Reported by @Yappo]

### Thanks

- cosmo0920
- Naoya Murakami
- Tetsuharu OHZEKI
- GMO Media, Inc.
- @Yappo

## Release 4.0.1 - 2014-03-29

### Improvements

- [doc] Added a link in return value to detailed header description (Output format).
- Supported to inspect vector and object value in JSON load. It shows more details about data which is failed to load.
- Added `adjuster` option to select command. adjuster options accepts following syntax: INDEX_COLUMN @ STRING_LITERAL (* FACTOR).
- Supported Weight vector column. You need to specify 'COLUMN_VECTOR|WITH_WEIGHT' flags to create weight vector column.
- Added missing MIN/MAX macros on SunOS. [GitHub#154] [Patch by Sebastian Wiedenroth]
- Improved recycling garbage data. It suppress to increse database size.
- [doc] Added documentation about GET parameters for groonga-suggest-httpd.
- [doc] Added documentation about Column.
- [doc] Added documentation about Vector column.
- [column_list] Supported to show weight vector column.
- [column_create] Added error check for creating multi column index without WITH_SECTION.
- [httpd] Enabled stub status module (NginxHttpStubStatusModule) for groonga-httpd. [Suggested by Masahiro Nagano]

### Fixes

- Fixed a bug that a division overflow caused a fatal error. For example, it occurs when you execute 'COLUMN / -1' operation to Int32 or Int64 column. [#2307]
- Fixed a bug that '%' operations performs '/' operations. [#2307]
- [doc] Fixed a wrong documentation about column_rename. [Reported by nise_nabe]
- Fixed the issue that out of bound array element access may occurs. [GitHub#158] [Reported by dcb314]

### Thanks

- Sebastian Wiedenroth
- Masahiro Nagano
- nise_nabe
- dcb314

## Release 4.0.0 - 2014-02-09

- Bump version to 4.0.0!

### Improvements

- [normalizer] Supported to show "checks" which is used for calculating next character position. Use WITH_CHECKS flag to enable this feature.
- [deb] Dropped Ubuntu 13.04 support.

### Fixes

- Fixed a crash bug that an object in grn_expr is used after it is freed. Normally Groonga server users aren't affected this bug. This bug mainly affects Rroonga users. Because this bug is occured by specifying column name including pseudo column name - such as '_key' - Rroonga users may use the usage.
- Fixed not to execute unexpected cascade delete which is introduced Groonga 3.0.8 release. If source's range and index's domain are different, Groonga doesn't execute cascade delete. [groonga-dev,02073] [Reported by yoku]
- Fixed not to publish grn_snip structure. Use grn_obj instead of grn_snip. If you use grn_snip_close, please replace grn_snip_close to grn_obj_close.
- [snippet_html] Fixed a crash bug when --query is empty. [groonga-dev,02097] [Reported by Naoya Murakami]
- [snippet_html] Fixed to suppress ALERT level message when contents of column is empty text. [groonga-dev,02097] [Reported by Naoya Murakami]
- [groonga-httpd] Fixed a bug "off" is used as path name in groonga_query_log_path. [groonga-dev,02113] [Reported by Ryoji Yamamoto]

### Thanks

- yoku
- Naoya Murakami
- Ryoji Yamamoto