fix(cache): fix cache to HASH

REFACTOR_SUMMARY.md

@@ -0,0 +1,143 @@
+# NoSQL 文档系统重构总结
+
+## 分支: 214-refactor-refactor-document-update
+
+### 完成的重构工作
+
+#### 1. 依赖升级
+- 升级了所有主要依赖项到最新版本
+- 包括 agones, firebase, aws-sdk-go-v2, nats.go, redis, mongo-driver 等
+- 确保安全性和功能的最新更新
+
+#### 2. 核心功能增強
+
+##### 2.1 延迟回写机制 (`document.go`, `worker.go`)
+- **新增 WriteBackPayload**: 定义回写消息格式
+- **新增 WriteBackOptions**: 配置回写行为
+- **新增 WriteBackWorker**: 处理异步回写操作
+- **增强 DocumentBase**: 支持延迟回写功能
+
+**特性:**
+- 支持延迟写入减少数据库压力
+- 基于消息队列的异步处理
+- 版本控制避免并发冲突
+- 完善的错误处理和重试机制
+
+##### 2.2 公共工具函数 (`common.go`)
+- **isBasicType()**: 检查 Go 基本类型
+- **marshalAnyMap()**: 处理 Redis HASH 存储序列化
+- **map2StructShallow()**: map 到结构体转换（支持 JSON 反序列化）
+- **struct2MapShallow()**: 结构体到 map 转换
+- **diffMapAny()**: 比较两个 map 并返回差异
+
+**改进:**
+- 更健壮的类型转换处理
+- 支持 []byte 和 string 类型的 JSON 数据
+- 完善的错误处理
+
+##### 2.3 配置管理 (`config.go`)
+- **WriteBackConfig**: 完整的回写配置结构
+- 支持环境变量配置
+- 内置验证机制
+- 默认值设定
+
+**配置项:**
+- `Enabled`: 是否启用回写
+- `Delay`: 回写延迟时间
+- `BatchSize`: 批处理大小
+- `MaxRetries`: 最大重试次数
+- `RetryDelay`: 重试延迟
+- `WorkerCount`: 工作器数量
+- `QueueSize`: 队列大小
+
+##### 2.4 管理器模式 (`manager.go`)
+- **WriteBackManager**: 管理多个回写工作器
+- **WriteBackManagerMetrics**: 详细的指标统计
+- 支持动态扩缩容
+- 健康检查机制
+- 优雅关闭
+
+#### 3. 测试完善
+
+##### 3.1 单元测试
+- `writeback_test.go`: 回写功能测试
+- `config_test.go`: 配置验证测试
+- `common_test.go`: 公共函数测试（原有，已修复）
+
+##### 3.2 Mock 实现
+- MockMessageQueue: 消息队列模拟
+- MockDocumentProvider: 文档提供者模拟
+- MockCollection: 集合模拟
+- MockSubscription: 订阅模拟
+
+#### 4. 消息队列增强 (`mq/miface/sub_options.go`)
+- 新增并发控制选项
+- 增强重试机制配置
+- 支持死信队列
+- 自动确认机制优化
+
+#### 5. 缓存系统改进 (`orm/nosql/cache/redis_cache.go`)
+- 改进 Redis HASH 存储支持
+- 更好的序列化/反序列化处理
+- 增强接口定义
+
+### 修复的问题
+
+1. **类型转换问题**: 修复了 `map2StructShallow` 中假设数据为 string 类型的错误
+2. **函数重复定义**: 移除了 `diffMapAny` 的重复实现
+3. **导入包问题**: 修复了各种未使用导入包的编译错误
+4. **接口匹配问题**: 修正了测试中的 Mock 对象接口实现
+
+### 新增的功能特性
+
+1. **异步写回**: 支持延迟写入数据库，提高性能
+2. **配置验证**: 完整的配置验证机制
+3. **指标监控**: 详细的运行时指标收集
+4. **健康检查**: 组件健康状态检查
+5. **优雅关闭**: 支持优雅停止所有工作器
+6. **错误分类**: 根据错误类型决定重试策略
+
+### 性能优化
+
+1. **写入性能**: 通过延迟回写减少实时写入压力
+2. **并发处理**: 支持多工作器并发处理
+3. **批处理**: 支持批量处理以提高效率
+4. **缓存优化**: 改进的 Redis 缓存实现
+
+### 监控和观察性
+
+1. **指标收集**: 处理数量、失败率、延迟时间等
+2. **日志记录**: 结构化日志，包含详细上下文
+3. **健康检查**: 组件状态监控
+4. **错误分类**: 不同错误类型的统计
+
+### 部署建议
+
+1. **渐进式部署**: 建议先在测试环境验证
+2. **配置调优**: 根据实际负载调整工作器数量和延迟时间
+3. **监控设置**: 配置相应的指标收集和告警
+4. **回滚准备**: 保持回滚到之前版本的能力
+
+### 下一步计划
+
+1. **性能测试**: 对比回写机制前后的性能差异
+2. **压力测试**: 验证高负载下的稳定性
+3. **文档完善**: 补充 API 文档和使用指南
+4. **示例代码**: 提供完整的使用示例
+
+### 风险评估
+
+**低风险:**
+- 向后兼容性良好
+- 新功能可选择性启用
+- 完善的测试覆盖
+
+**需要注意:**
+- 新的消息队列依赖
+- 配置复杂度增加
+- 运行时资源消耗
+
+**建议:**
+- 充分测试后再部署到生产环境
+- 监控资源使用情况
+- 准备降级方案

go.mod

@@ -3,78 +3,79 @@ module github.com/gstones/moke-kit
 go 1.24.2
 
 require (
-	agones.dev/agones v1.48.0
-	firebase.google.com/go/v4 v4.15.2
+	agones.dev/agones v1.49.0
+	firebase.google.com/go/v4 v4.16.1
 	github.com/ThreeDotsLabs/watermill v1.4.6
 	github.com/ThreeDotsLabs/watermill-nats/v2 v2.1.3
 	github.com/abiosoft/ishell v2.0.0+incompatible
-	github.com/awa/go-iap v1.42.1
-	github.com/aws/aws-sdk-go-v2 v1.36.3
-	github.com/aws/aws-sdk-go-v2/config v1.29.14
-	github.com/aws/aws-sdk-go-v2/credentials v1.17.67
-	github.com/grpc-ecosystem/go-grpc-middleware/v2 v2.3.1
+	github.com/awa/go-iap v1.43.0
+	github.com/aws/aws-sdk-go-v2 v1.36.4
+	github.com/aws/aws-sdk-go-v2/config v1.29.16
+	github.com/aws/aws-sdk-go-v2/credentials v1.17.69
+	github.com/grpc-ecosystem/go-grpc-middleware/v2 v2.3.2
 	github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.3
 	github.com/gstones/zinx v1.2.7-0.20240617071724-88bd884d8d08
-	github.com/hashicorp/vault/api v1.16.0
+	github.com/hashicorp/vault/api v1.20.0
 	github.com/kelseyhightower/envconfig v1.4.0
-	github.com/nats-io/nats.go v1.41.1
+	github.com/nats-io/nats.go v1.43.0
 	github.com/pkg/errors v0.9.1
-	github.com/redis/go-redis/v9 v9.7.3
+	github.com/redis/go-redis/v9 v9.10.0
 	github.com/soheilhy/cmux v0.1.5
 	github.com/stretchr/testify v1.10.0
 	github.com/supabase-community/supabase-go v0.0.4
-	go.mongodb.org/mongo-driver v1.17.3
-	go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.60.0
-	go.opentelemetry.io/otel v1.35.0
-	go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.35.0
-	go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.35.0
-	go.opentelemetry.io/otel/sdk v1.35.0
-	go.opentelemetry.io/otel/sdk/metric v1.35.0
-	go.opentelemetry.io/otel/trace v1.35.0
+	go.mongodb.org/mongo-driver v1.17.4
+	go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.61.0
+	go.opentelemetry.io/otel v1.36.0
+	go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.36.0
+	go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.36.0
+	go.opentelemetry.io/otel/sdk v1.36.0
+	go.opentelemetry.io/otel/sdk/metric v1.36.0
+	go.opentelemetry.io/otel/trace v1.36.0
 	go.uber.org/atomic v1.11.0
 	go.uber.org/automaxprocs v1.6.0
-	go.uber.org/fx v1.23.0
+	go.uber.org/fx v1.24.0
 	go.uber.org/zap v1.27.0
-	golang.org/x/exp v0.0.0-20250408133849-7e4ce0ab07d0
-	golang.org/x/net v0.39.0
-	golang.org/x/time v0.11.0
-	google.golang.org/api v0.228.0
-	google.golang.org/grpc v1.71.1
+	golang.org/x/exp v0.0.0-20250606033433-dcc06ee1d476
+	golang.org/x/net v0.41.0
+	golang.org/x/time v0.12.0
+	google.golang.org/api v0.237.0
+	google.golang.org/grpc v1.73.0
 	gopkg.in/fsnotify.v1 v1.4.7
-	gorm.io/gorm v1.25.12
+	gorm.io/gorm v1.30.0
 )
 
 require (
-	cel.dev/expr v0.23.1 // indirect
-	cloud.google.com/go v0.120.0 // indirect
-	cloud.google.com/go/auth v0.15.0 // indirect
+	cel.dev/expr v0.24.0 // indirect
+	cloud.google.com/go v0.121.2 // indirect
+	cloud.google.com/go/auth v0.16.2 // indirect
 	cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect
-	cloud.google.com/go/compute/metadata v0.6.0 // indirect
+	cloud.google.com/go/compute/metadata v0.7.0 // indirect
 	cloud.google.com/go/firestore v1.18.0 // indirect
-	cloud.google.com/go/iam v1.5.0 // indirect
-	cloud.google.com/go/longrunning v0.6.6 // indirect
-	cloud.google.com/go/monitoring v1.24.1 // indirect
-	cloud.google.com/go/storage v1.51.0 // indirect
-	github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.27.0 // indirect
-	github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.51.0 // indirect
-	github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.51.0 // indirect
+	cloud.google.com/go/iam v1.5.2 // indirect
+	cloud.google.com/go/longrunning v0.6.7 // indirect
+	cloud.google.com/go/monitoring v1.24.2 // indirect
+	cloud.google.com/go/storage v1.55.0 // indirect
+	github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.28.0 // indirect
+	github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.52.0 // indirect
+	github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.52.0 // indirect
 	github.com/MicahParks/keyfunc v1.9.0 // indirect
 	github.com/abiosoft/readline v0.0.0-20180607040430-155bce2042db // indirect
-	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.30 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.34 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.34 // indirect
+	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.31 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.35 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.35 // indirect
 	github.com/aws/aws-sdk-go-v2/internal/ini v1.8.3 // indirect
 	github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.3 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.15 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sso v1.25.3 // indirect
-	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.30.1 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sts v1.33.19 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.16 // indirect
+	github.com/aws/aws-sdk-go-v2/service/sso v1.25.4 // indirect
+	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.30.2 // indirect
+	github.com/aws/aws-sdk-go-v2/service/sts v1.33.21 // indirect
 	github.com/aws/smithy-go v1.22.3 // indirect
 	github.com/cenkalti/backoff/v4 v4.3.0 // indirect
+	github.com/cenkalti/backoff/v5 v5.0.2 // indirect
 	github.com/cespare/xxhash/v2 v2.3.0 // indirect
 	github.com/chzyer/logex v1.1.10 // indirect
 	github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1 // indirect
-	github.com/cncf/xds/go v0.0.0-20250326154945-ae57f3c0d45f // indirect
+	github.com/cncf/xds/go v0.0.0-20250501225837-2ac532fd4443 // indirect
 	github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
 	github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
 	github.com/envoyproxy/go-control-plane/envoy v1.32.4 // indirect
@@ -84,7 +85,7 @@ require (
 	github.com/flynn-archive/go-shlex v0.0.0-20150515145356-3f9db97f8568 // indirect
 	github.com/fsnotify/fsnotify v1.9.0 // indirect
 	github.com/go-jose/go-jose/v4 v4.1.0 // indirect
-	github.com/go-logr/logr v1.4.2 // indirect
+	github.com/go-logr/logr v1.4.3 // indirect
 	github.com/go-logr/stdr v1.2.2 // indirect
 	github.com/go-test/deep v1.1.0 // indirect
 	github.com/golang-jwt/jwt/v4 v4.5.2 // indirect
@@ -94,7 +95,7 @@ require (
 	github.com/google/s2a-go v0.1.9 // indirect
 	github.com/google/uuid v1.6.0 // indirect
 	github.com/googleapis/enterprise-certificate-proxy v0.3.6 // indirect
-	github.com/googleapis/gax-go/v2 v2.14.1 // indirect
+	github.com/googleapis/gax-go/v2 v2.14.2 // indirect
 	github.com/gorilla/websocket v1.5.3 // indirect
 	github.com/hashicorp/errwrap v1.1.0 // indirect
 	github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
@@ -116,12 +117,13 @@ require (
 	github.com/mitchellh/go-homedir v1.1.0 // indirect
 	github.com/mitchellh/mapstructure v1.5.0 // indirect
 	github.com/montanaflynn/stats v0.7.1 // indirect
-	github.com/nats-io/nkeys v0.4.10 // indirect
+	github.com/nats-io/nkeys v0.4.11 // indirect
 	github.com/nats-io/nuid v1.0.1 // indirect
 	github.com/oklog/ulid v1.3.1 // indirect
 	github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 // indirect
 	github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
 	github.com/ryanuber/go-glob v1.0.0 // indirect
+	github.com/spiffe/go-spiffe/v2 v2.5.0 // indirect
 	github.com/stretchr/objx v0.5.2 // indirect
 	github.com/supabase-community/functions-go v0.1.0 // indirect
 	github.com/supabase-community/gotrue-go v1.2.1 // indirect
@@ -136,23 +138,24 @@ require (
 	github.com/xdg-go/stringprep v1.0.4 // indirect
 	github.com/xtaci/kcp-go v5.4.20+incompatible // indirect
 	github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78 // indirect
+	github.com/zeebo/errs v1.4.0 // indirect
 	go.opentelemetry.io/auto/sdk v1.1.0 // indirect
-	go.opentelemetry.io/contrib/detectors/gcp v1.35.0 // indirect
-	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.60.0 // indirect
-	go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.35.0 // indirect
-	go.opentelemetry.io/otel/metric v1.35.0 // indirect
-	go.opentelemetry.io/proto/otlp v1.5.0 // indirect
-	go.uber.org/dig v1.18.1 // indirect
+	go.opentelemetry.io/contrib/detectors/gcp v1.36.0 // indirect
+	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 // indirect
+	go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.36.0 // indirect
+	go.opentelemetry.io/otel/metric v1.36.0 // indirect
+	go.opentelemetry.io/proto/otlp v1.7.0 // indirect
+	go.uber.org/dig v1.19.0 // indirect
 	go.uber.org/multierr v1.11.0 // indirect
-	golang.org/x/crypto v0.37.0 // indirect
-	golang.org/x/oauth2 v0.29.0 // indirect
-	golang.org/x/sync v0.13.0 // indirect
-	golang.org/x/sys v0.32.0 // indirect
-	golang.org/x/text v0.24.0 // indirect
+	golang.org/x/crypto v0.39.0 // indirect
+	golang.org/x/oauth2 v0.30.0 // indirect
+	golang.org/x/sync v0.15.0 // indirect
+	golang.org/x/sys v0.33.0 // indirect
+	golang.org/x/text v0.26.0 // indirect
 	google.golang.org/appengine/v2 v2.0.6 // indirect
-	google.golang.org/genproto v0.0.0-20250409194420-de1ac958c67a // indirect
-	google.golang.org/genproto/googleapis/api v0.0.0-20250409194420-de1ac958c67a // indirect
-	google.golang.org/genproto/googleapis/rpc v0.0.0-20250409194420-de1ac958c67a // indirect
+	google.golang.org/genproto v0.0.0-20250603155806-513f23925822 // indirect
+	google.golang.org/genproto/googleapis/api v0.0.0-20250603155806-513f23925822 // indirect
+	google.golang.org/genproto/googleapis/rpc v0.0.0-20250603155806-513f23925822 // indirect
 	google.golang.org/protobuf v1.36.6 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
 )