Skip to content

fix(function): bound group_concat result and clamp stream out columns#35408

Open
bestdo77 wants to merge 1 commit into
taosdata:mainfrom
bestdo77:fix/gconcat-overflow-stream-34504
Open

fix(function): bound group_concat result and clamp stream out columns#35408
bestdo77 wants to merge 1 commit into
taosdata:mainfrom
bestdo77:fix/gconcat-overflow-stream-34504

Conversation

@bestdo77

Copy link
Copy Markdown

Fixes #34504.

group_concat has two problems:

  1. The result is accumulated into a fixed 65519-byte buffer with no bounds check (gconcatHelper in builtinsimpl.c). Once a group's payload grows past that, the memcpy corrupts the heap and taosd aborts (malloc(): corrupted top size, signal 6). I reproduced this on current main by running group_concat over ~6k rows (~102 KB payload) — the server dies immediately.

  2. translateGroupConcat always estimates the result width as 65515, so a stream output row containing any other column exceeds the 65531 limit and CREATE STREAM fails with Row length exceeds max length 65531 [0x8000263E] — the error from the issue.

What this PR does:

  • gconcatHelper now checks the length before every append and returns "Value too long for column/tag" when the payload would exceed the limit. The query fails, the server stays up. A cleanupFunc is registered for group_concat and all buffer frees are idempotent, so the error path releases the accumulation buffer instead of leaking it.
  • When the auto-derived stream output row is too wide, group_concat output columns are clamped to what fits (createStreamReqSetDefaultOutCols). The stream in the issue can be created now. If a real value is still longer than the clamped column, the write is rejected with a normal error instead of crashing.

Reproduced/tested with:

-- 600+ rows of 120+ byte values: before = server aborts, after = clean error
select group_concat(v, ',') from t;

-- before = 0x8000263E row too long, after = created
create stream s1 count_window(10) from s_ptr_data partition by tbname into output as
  select last(ts) as ts, avg(test_value) as avg_test_value, group_concat(test_value_str, ',') as test_value_array
  from s_ptr_data;

Added test_group_concat_overflow to test_agg_gconcat.py covering both cases.

One note for the reporter: the SQL in the issue has no window constraint (ts >= _twstart and ts <= _twend), so each trigger scans the whole table — that is how the calc query behaves without it; the stream cases in this repo always include it.

Tested on Debian 10 x86_64, community build 3.4.1.13.alpha based on main@df445db5:

  • parserTest stream suite passes (30/30)
  • test_agg_gconcat.py 3/3, test_agg_smoking.py 6/6
  • on a live node the overflow query returns 0x80002653 in ~10 ms and the server keeps serving; the stream above is created (output row 65533 -> 65531 after the clamp)

Performance impact should be negligible: one integer comparison per appended value in group_concat, and the clamp runs once per stream creation only when the row is oversized. No other code paths change. The only behavior change is that an oversized group_concat now returns an error instead of crashing the server.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds bounds checking to prevent buffer overflow in group_concat (returning TSDB_CODE_PAR_VALUE_TOO_LONG if the limit is exceeded), registers a cleanup function to prevent memory leaks, and adds integration tests for overflow safety. The review feedback points out a potential null pointer dereference of pRes in the error handling block of gconcatFunction when code != TSDB_CODE_SUCCESS.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread source/libs/function/src/builtinsimpl.c
@bestdo77
bestdo77 force-pushed the fix/gconcat-overflow-stream-34504 branch from ab4f331 to 03be346 Compare July 17, 2026 08:12
@bestdo77 bestdo77 changed the title fix: bound group_concat result and clamp stream out columns fix(function): bound group_concat result and clamp stream out columns Jul 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3.4.0.2版本中在计数窗口流计算中使用group_concat聚合函数会引发宕机和报错

1 participant