341 lines
6.7 KiB
Markdown
341 lines
6.7 KiB
Markdown
|
# 导入速度慢修复说明
|
|||
|
|
|
|||
|
|
## ❌ **问题现象**
|
|||
|
|
|
|||
|
|
导入100条数据需要2-3分钟，比之前慢了好几倍。
|
|||
|
|
|
|||
|
|
### **日志分析**
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
slow sql 1164 millis. update sys_user (每条1.2秒)
|
|||
|
|
slow sql 1165 millis. update student_class (每条1.2秒)
|
|||
|
|
Duplicate entry '234-17' for key 'student_class.uk_student_class'
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 🔍 **问题原因**
|
|||
|
|
|
|||
|
|
### **原因1：班级分配逻辑错误**
|
|||
|
|
|
|||
|
|
**错误逻辑：**
|
|||
|
|
1. 将旧班级设置为 `status=0`
|
|||
|
|
2. 插入新班级记录
|
|||
|
|
|
|||
|
|
**问题：**
|
|||
|
|
- 如果新旧班级ID相同，INSERT失败（Duplicate）
|
|||
|
|
- 每次都UPDATE然后INSERT，即使班级未变
|
|||
|
|
|
|||
|
|
### **原因2：数据库缺少索引**
|
|||
|
|
|
|||
|
|
- `sys_user.user_name` 无索引
|
|||
|
|
- `student_class.student_id` 无索引
|
|||
|
|
- 导致查询很慢（1.2秒/条）
|
|||
|
|
|
|||
|
|
### **原因3：重复的数据库操作**
|
|||
|
|
|
|||
|
|
- 每条记录都查询班级列表
|
|||
|
|
- 每条记录都UPDATE status
|
|||
|
|
- 没有必要的缓存
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## ✅ **已修复内容**
|
|||
|
|
|
|||
|
|
### **修复1：优化班级分配逻辑**
|
|||
|
|
|
|||
|
|
**文件：** `StudyClassUserServiceImpl.java` (第1119-1181行)
|
|||
|
|
|
|||
|
|
**新逻辑：**
|
|||
|
|
```java
|
|||
|
|
// 1. 检查是否已在该班级
|
|||
|
|
if (已在该班级) {
|
|||
|
|
if (状态不是活跃) {
|
|||
|
|
// 只需UPDATE状态
|
|||
|
|
UPDATE status = 1
|
|||
|
|
}
|
|||
|
|
// 否则什么都不做
|
|||
|
|
}
|
|||
|
|
else {
|
|||
|
|
// 2. 禁用其他班级
|
|||
|
|
UPDATE 其他班级 status = 0
|
|||
|
|
|
|||
|
|
// 3. 插入新班级
|
|||
|
|
INSERT 新班级记录
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
**优势：**
|
|||
|
|
- ✅ 避免Duplicate错误
|
|||
|
|
- ✅ 减少不必要的UPDATE
|
|||
|
|
- ✅ 班级未变时不做任何操作
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### **修复2：创建数据库优化SQL**
|
|||
|
|
|
|||
|
|
**文件：** `log/Sql/optimize_import_performance.sql`
|
|||
|
|
|
|||
|
|
**包含：**
|
|||
|
|
- 添加必要的索引
|
|||
|
|
- 优化表统计信息
|
|||
|
|
- 性能监控查询
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 🚀 **部署步骤**
|
|||
|
|
|
|||
|
|
### **步骤1：重新编译后端**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
cd C:\Users\Administrator\Desktop\Project\ry_study-v_03\Study-Vue-redis
|
|||
|
|
mvn clean package -DskipTests
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### **步骤2：执行数据库优化SQL**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# 连接数据库
|
|||
|
|
mysql -u root -p ry_study
|
|||
|
|
|
|||
|
|
# 执行优化SQL
|
|||
|
|
source C:/Users/Administrator/Desktop/Project/ry_study-v_03/log/Sql/optimize_import_performance.sql
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
**或者在Navicat中：**
|
|||
|
|
1. 打开数据库：`ry_study`
|
|||
|
|
2. 打开查询窗口
|
|||
|
|
3. 粘贴 `optimize_import_performance.sql` 内容
|
|||
|
|
4. 执行
|
|||
|
|
|
|||
|
|
### **步骤3：重启服务**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# 停止旧服务
|
|||
|
|
# 启动新服务
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### **步骤4：测试导入**
|
|||
|
|
|
|||
|
|
1. 准备测试数据（100条）
|
|||
|
|
2. 执行导入
|
|||
|
|
3. 记录时间
|
|||
|
|
4. 查看日志
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 📊 **性能对比**
|
|||
|
|
|
|||
|
|
### **修复前**
|
|||
|
|
|
|||
|
|
| 项目 | 数值 |
|
|||
|
|
|------|------|
|
|||
|
|
| 100条导入时间 | 2-3分钟 |
|
|||
|
|
| 每条平均时间 | 1.2-1.8秒 |
|
|||
|
|
| Duplicate错误 | 频繁出现 |
|
|||
|
|
| 慢查询 | 大量 |
|
|||
|
|
|
|||
|
|
### **修复后（预期）**
|
|||
|
|
|
|||
|
|
| 项目 | 数值 |
|
|||
|
|
|------|------|
|
|||
|
|
| 100条导入时间 | 10-30秒 |
|
|||
|
|
| 每条平均时间 | 0.1-0.3秒 |
|
|||
|
|
| Duplicate错误 | 无 |
|
|||
|
|
| 慢查询 | 基本无 |
|
|||
|
|
|
|||
|
|
**性能提升：5-10倍**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 🔧 **添加的索引**
|
|||
|
|
|
|||
|
|
```sql
|
|||
|
|
-- sys_user 表
|
|||
|
|
CREATE INDEX idx_user_name ON sys_user(user_name); -- 快速按信息编号查询
|
|||
|
|
CREATE INDEX idx_nick_name ON sys_user(nick_name); -- 快速按姓名查询
|
|||
|
|
CREATE INDEX idx_prison_area ON sys_user(prison_area); -- 快速按监区查询
|
|||
|
|
|
|||
|
|
-- student_class 表
|
|||
|
|
CREATE INDEX idx_student_id ON student_class(student_id); -- 快速按学员查询
|
|||
|
|
CREATE INDEX idx_class_id ON student_class(class_id); -- 快速按班级查询
|
|||
|
|
CREATE INDEX idx_status ON student_class(status); -- 快速按状态查询
|
|||
|
|
CREATE INDEX idx_student_status ON student_class(student_id, status); -- 复合索引
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 🧪 **测试验证**
|
|||
|
|
|
|||
|
|
### **测试1：导入速度**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# 1. 生成测试数据
|
|||
|
|
python generate_test_data.py
|
|||
|
|
|
|||
|
|
# 2. 记录开始时间
|
|||
|
|
开始时间: 14:00:00
|
|||
|
|
|
|||
|
|
# 3. 导入test_data.xlsx (100条)
|
|||
|
|
|
|||
|
|
# 4. 记录结束时间
|
|||
|
|
结束时间: 14:00:15
|
|||
|
|
|
|||
|
|
# 5. 计算耗时
|
|||
|
|
耗时: 15秒（之前需要2-3分钟）
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### **测试2：无Duplicate错误**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# 查看日志
|
|||
|
|
grep "Duplicate entry" logs/app.log
|
|||
|
|
|
|||
|
|
# 预期结果：无结果
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### **测试3：无慢查询**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# 查看日志
|
|||
|
|
grep "slow sql" logs/app.log
|
|||
|
|
|
|||
|
|
# 预期结果：无结果或很少
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 📋 **检查清单**
|
|||
|
|
|
|||
|
|
### **代码修复**
|
|||
|
|
- [x] 修复班级分配逻辑
|
|||
|
|
- [x] 避免Duplicate错误
|
|||
|
|
- [x] 减少不必要的UPDATE
|
|||
|
|
|
|||
|
|
### **数据库优化**
|
|||
|
|
- [ ] 执行优化SQL
|
|||
|
|
- [ ] 添加所有索引
|
|||
|
|
- [ ] 更新表统计信息
|
|||
|
|
|
|||
|
|
### **部署验证**
|
|||
|
|
- [ ] 重新编译后端
|
|||
|
|
- [ ] 重启服务
|
|||
|
|
- [ ] 测试导入100条数据
|
|||
|
|
- [ ] 查看日志确认无错误
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 💡 **进一步优化建议**
|
|||
|
|
|
|||
|
|
### **1. 批量操作**
|
|||
|
|
- 批量INSERT（每批50条）
|
|||
|
|
- 批量UPDATE
|
|||
|
|
- 减少数据库往返次数
|
|||
|
|
|
|||
|
|
### **2. 使用缓存**
|
|||
|
|
- Redis缓存班级映射
|
|||
|
|
- 缓存角色ID
|
|||
|
|
- 减少重复查询
|
|||
|
|
|
|||
|
|
### **3. 异步处理**
|
|||
|
|
- 使用消息队列
|
|||
|
|
- 后台异步导入
|
|||
|
|
- 提高并发性
|
|||
|
|
|
|||
|
|
### **4. 连接池优化**
|
|||
|
|
```yaml
|
|||
|
|
spring:
|
|||
|
|
datasource:
|
|||
|
|
druid:
|
|||
|
|
max-active: 50 # 增加最大连接数
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## ⚠️ **注意事项**
|
|||
|
|
|
|||
|
|
### **1. 数据库备份**
|
|||
|
|
- 执行SQL前务必备份数据库
|
|||
|
|
- 建议先在测试环境验证
|
|||
|
|
|
|||
|
|
### **2. 索引维护**
|
|||
|
|
- 索引会占用磁盘空间
|
|||
|
|
- 写入性能可能略有下降（可忽略）
|
|||
|
|
- 定期ANALYZE TABLE
|
|||
|
|
|
|||
|
|
### **3. 监控**
|
|||
|
|
- 持续监控慢查询日志
|
|||
|
|
- 关注数据库连接数
|
|||
|
|
- 关注内存使用
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 🆘 **常见问题**
|
|||
|
|
|
|||
|
|
### **Q1：索引创建失败**
|
|||
|
|
```sql
|
|||
|
|
-- 检查是否已存在同名索引
|
|||
|
|
SHOW INDEX FROM sys_user WHERE Key_name = 'idx_user_name';
|
|||
|
|
|
|||
|
|
-- 删除后重新创建
|
|||
|
|
DROP INDEX idx_user_name ON sys_user;
|
|||
|
|
CREATE INDEX idx_user_name ON sys_user(user_name);
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### **Q2：仍然出现Duplicate错误**
|
|||
|
|
- 检查代码是否重新编译
|
|||
|
|
- 检查服务是否重启
|
|||
|
|
- 查看日志确认使用新代码
|
|||
|
|
|
|||
|
|
### **Q3：性能没有明显提升**
|
|||
|
|
- 检查索引是否创建成功
|
|||
|
|
- 执行ANALYZE TABLE
|
|||
|
|
- 查看EXPLAIN确认使用了索引
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## ✅ **验收标准**
|
|||
|
|
|
|||
|
|
- [ ] 100条数据导入时间 < 30秒
|
|||
|
|
- [ ] 无Duplicate错误
|
|||
|
|
- [ ] 无slow sql警告（或很少）
|
|||
|
|
- [ ] 日志无异常
|
|||
|
|
- [ ] 班级分配正确
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 📝 **修改文件清单**
|
|||
|
|
|
|||
|
|
### **后端代码**
|
|||
|
|
1. ✅ `StudyClassUserServiceImpl.java` - 修复班级分配逻辑
|
|||
|
|
|
|||
|
|
### **数据库**
|
|||
|
|
2. ✅ `optimize_import_performance.sql` - 添加索引和优化
|
|||
|
|
|
|||
|
|
### **文档**
|
|||
|
|
3. ✅ `导入性能优化方案.md` - 详细分析
|
|||
|
|
4. ✅ `导入速度慢修复说明.md` - 修复说明（本文档）
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 🎯 **总结**
|
|||
|
|
|
|||
|
|
### **核心问题**
|
|||
|
|
1. 班级分配逻辑错误 → Duplicate错误
|
|||
|
|
2. 缺少数据库索引 → 慢查询
|
|||
|
|
3. 重复的数据库操作 → 性能差
|
|||
|
|
|
|||
|
|
### **解决方案**
|
|||
|
|
1. 修复班级分配逻辑
|
|||
|
|
2. 添加必要的索引
|
|||
|
|
3. 优化数据库配置
|
|||
|
|
|
|||
|
|
### **预期效果**
|
|||
|
|
- **性能提升：5-10倍**
|
|||
|
|
- **无错误：无Duplicate**
|
|||
|
|
- **体验更好：导入更流畅**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**现在请按照步骤执行修复，完成后测试导入速度！** 🚀
|