[lightning] TiDB Lightning can report success after pagination skips replace conflicts

[wjhuang2016]

Bug Report

1. Minimal reproduce step (Required)

This reproduces on TiDB/Lightning master
05b396fb6636f73b3bc06b09107cf43f2c725c35 with one TiDB, one PD, real TiKV,
MDL enabled, strict SQL mode, the local backend, and
conflict.strategy = "replace".

Set LIGHTNING to the current tidb-lightning binary and run:

set -euo pipefail

LIGHTNING=${LIGHTNING:-tidb-lightning}
MYSQL=${MYSQL:-mysql}
HOST=${HOST:-127.0.0.1}
PORT=${PORT:-4000}
STATUS_PORT=${STATUS_PORT:-10080}
PD=${PD:-127.0.0.1:2379}
DB=lightning_page_repro
TASK_DB=lightning_page_repro_tasks
WORK=$(mktemp -d /tmp/lightning-page-repro.XXXXXX)
DATA_A=$WORK/data-a
DATA_B=$WORK/data-b
mkdir -p "$DATA_A" "$DATA_B"

mysql_cmd=("$MYSQL" -h"$HOST" -P"$PORT" -uroot -N -B)
"${mysql_cmd[@]}" -e "
DROP DATABASE IF EXISTS \`$DB\`;
DROP DATABASE IF EXISTS \`$TASK_DB\`;"

for dir in "$DATA_A" "$DATA_B"; do
  printf 'CREATE DATABASE `%s`;\n' "$DB" >"$dir/$DB-schema-create.sql"
  cat >"$dir/$DB.accounts-schema.sql" <<'SQL'
CREATE TABLE accounts (
  id BIGINT PRIMARY KEY CLUSTERED,
  u BIGINT NOT NULL,
  payload VARCHAR(64) NOT NULL,
  UNIQUE KEY uk_u(u)
);
SQL
done

# Task A leaves exactly 999 retained index-conflict rows.
printf 'id,u,payload\n' >"$DATA_A/$DB.accounts.0.csv"
for i in $(seq 1 999); do
  if (( i <= 996 )); then
    u=$(((i + 1) / 2))
  else
    u=499
  fi
  printf '%d,%d,history-%04d\n' "$i" "$u" "$i" \
    >>"$DATA_A/$DB.accounts.0.csv"
done

cat >"$DATA_B/$DB.accounts.0.csv" <<'CSV'
id,u,payload
2001,1001,current-first
2002,1001,current-second
CSV

write_config() {
  path=$1
  data_dir=$2
  sorted_dir=$3
  cat >"$path" <<TOML
[lightning]
level = "info"
region-concurrency = 1
task-info-schema-name = "$TASK_DB"

[tidb]
host = "$HOST"
port = $PORT
user = "root"
status-port = $STATUS_PORT
pd-addr = "$PD"

[tikv-importer]
backend = "local"
sorted-kv-dir = "$sorted_dir"
add-index-by-sql = false

[mydumper]
data-source-dir = "$data_dir"
batch-size = 1

[mydumper.csv]
header = true

[conflict]
strategy = "replace"

[checkpoint]
enable = true
TOML
}

write_config "$WORK/a.toml" "$DATA_A" "$WORK/sorted-a"
write_config "$WORK/b.toml" "$DATA_B" "$WORK/sorted-b"

"$LIGHTNING" -config "$WORK/a.toml" \
  -status-addr 127.0.0.1:18331 -log-file "$WORK/a.log"

# Recreate only the application database. Retain task-info rows for audit.
"${mysql_cmd[@]}" -e "DROP DATABASE \`$DB\`;"

"$LIGHTNING" -config "$WORK/b.toml" \
  -status-addr 127.0.0.1:18332 -log-file "$WORK/b.log"

"${mysql_cmd[@]}" -e "
SELECT COUNT(*) AS primary_rows FROM \`$DB\`.accounts FORCE INDEX(PRIMARY);
SELECT COUNT(*) AS unique_index_rows FROM \`$DB\`.accounts FORCE INDEX(uk_u);
SELECT id,u,payload FROM \`$DB\`.accounts ORDER BY id;
ADMIN CHECK TABLE \`$DB\`.accounts;"

Both Lightning commands log:

[resolve-dupe] resolve duplicate rows completed
restore all tables data completed
tidb lightning exit successfully

The final checks show two record KVs but only one unique-index KV:

primary_rows      = 2
unique_index_rows = 1
2001  1001  current-first
2002  1001  current-second
ERROR 8223: data inconsistency in table: accounts, index: uk_u,
handle 2002 has no matching index value

Changing Task A from 999 to 998 conflict rows keeps both current conflict rows
in the first page; the result is one record, one index entry, and
ADMIN CHECK TABLE passes.

2. What did you expect to see? (Required)

Pagination should continue until the input query is exhausted. Retained rows
from an earlier task must not make the current resolver stop before processing
all current conflict rows. A successful Lightning run must leave record and
index ownership consistent.

3. What did you see instead? (Required)

An effect-free page makes the resolver stop even though more input rows exist.
Lightning reports successful duplicate resolution and exits 0 with persistent
record/index corruption.

4. What is your TiDB version? (Required)

TiDB/Lightning master 05b396fb6636f73b3bc06b09107cf43f2c725c35.

Likely root cause and fix direction

conflict_error_v4 stores task_id, but ReplaceConflictKeys queries rows by
table name and KV type without scoping them to the current task. Its page size
is 1000, and both conflict loops use:

if len(handleKeys) == 0 {
    break
}

handleKeys is the effect produced by a page, not proof that the input is
exhausted. The loop should count rows read, stop only on an empty input page,
advance the cursor after every nonempty page, and scope resolution to the
current task_id.