作者简介:靳昊达,瑞蓝创数据库工程师
原创内容未经授权不得随意使用,转载请联系小编并注明来源
一、创建基表、MLOG 与增量刷新物化视图
1.1 创建基表
创建两张业务表 orders(订单表)和 items(商品维度表),构成经典的订单-商品关联模型:
-- 订单表(50000+ 行)
CREATETABLE orders ( order_id BIGINT PRIMARY KEY, item_id BIGINTNOTNULL, price DECIMAL(10,2), amount DECIMAL(12,2), order_date DATE, region VARCHAR(50));-- 商品维度表(3 行)CREATETABLE items ( item_id BIGINT PRIMARY KEY, item_name VARCHAR(200),categoryVARCHAR(100), price DECIMAL(10,2));
1.2 创建物化视图日志(MLOG)
为两张基表分别创建 MLOG,关键参数:
-
WITH PRIMARY KEY, ROWID, SEQUENCE (...)— 记录主键、行ID、序列号及指定列的变更 -
INCLUDING NEW VALUES— 支持增量刷新时记录新值
CREATE MATERIALIZEDVIEWLOGON orders
WITH PRIMARY KEY, ROWID, SEQUENCE (item_id, amount, order_date, region)
INCLUDINGNEWVALUES;
CREATEMATERIALIZEDVIEWLOGON items
WITH PRIMARY KEY, ROWID, SEQUENCE (item_name, category, price)
INCLUDINGNEWVALUES;
1.3 创建增量刷新物化视图(每分钟自动刷新)
CREATE MATERIALIZEDVIEW mv_orders_items
REFRESHFASTONDEMAND
STARTWITHNOW() NEXTNOW() + INTERVAL1MINUTE
AS
SELECT
o.order_id, o.item_id, o.order_date, o.region,
i.item_name, i.category, i.price, o.amount,
COUNT(*) AS cnt
FROM orders o
JOIN items i ON o.item_id = i.item_id
GROUPBY o.order_id, o.item_id, o.order_date, o.region,
i.item_name, i.category, i.price, o.amount;
关键语法解读:
-
REFRESH FAST— 增量刷新(仅应用 MLOG 中的变更) -
ON DEMAND— 按需刷新(由调度 JOB 触发,非自动提交时刷新) -
START WITH NOW() NEXT NOW() + INTERVAL 1 MINUTE— 立即开始,每分钟调度一次 -
包含 JOIN + GROUP BY + COUNT(*),满足 FAST 刷新条件
二、物化视图运行状态验证(6 个诊断维度)
2.1 确认物化视图基本信息(dba_mviews)
SELECT mview_name, owner, rewrite_enabled, refresh_mode, refresh_method,
last_refresh_type, last_refresh_date, staleness
FROM oceanbase.dba_mviews
WHERE mview_name = 'MV_ORDERS_ITEMS';
输出结果:
|
|
|
|
|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2.2 查看刷新运行统计(dba_mvref_run_stats)
SELECT run_owner, mviews, refresh_id, method, num_mvs,
start_time, end_time, elapsed_time, parallelism,
number_of_failures, complete_stats_available
FROM oceanbase.dba_mvref_run_stats
WHERE mviews LIKE '%MV_ORDERS_ITEMS%'
ORDER BY start_time DESC;
输出结果(6 次连续自动刷新记录):
|
|
|
|
|
|
|
|
|---|---|---|---|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
关键发现:
-
每分钟自动刷新一次,method 为 NULL(自动调度不标记 method) -
parallelism = 4(默认并行度) -
elapsed_time = 0(空表数据量小,刷新耗时不到 1 秒) -
number_of_failures = 0,complete_stats_available = Y,全部成功
2.3 确认依赖关系(dba_mview_deps)
SELECT mview_owner, mview_name, dep_owner, dep_name, dep_type
FROM oceanbase.dba_mview_deps
WHERE mview_name = 'mv_orders_items';
|
|
|
|
|---|---|---|
|
|
|
|
|
|
|
|
2.4 查看正在刷新的物化视图(dba_mview_running_jobs)
SELECT svr_ip, svr_port, table_name, job_type, session_id,
parallel, job_start_time, read_snapshot
FROM oceanbase.dba_mview_running_jobs
WHERE table_name = 'mv_orders_items';
-- Empty set(当前无正在运行的刷新任务)
2.5 确认调度 JOB 信息(dba_scheduler_jobs)
SELECT owner, job_name, job_type, job_action,
repeat_interval, next_run_date, last_start_date,
state, enabled
FROM oceanbase.dba_scheduler_jobs
WHERE job_action LIKE '%mv_orders_items%';
|
|
|
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
关键发现:
-
JOB 名由系统自动生成,格式 MVIEW_REFRESH$J_ -
nested=>FALSE表示不级联刷新依赖的物化视图 -
调度间隔与创建时指定的 INTERVAL 1 MINUTE 一致
三、手动刷新并行度控制实验(3 种方法对比)
方法 1:设置系统变量 mview_refresh_dop
-- 查看默认值
SHOW VARIABLES LIKE "mview_refresh_dop";
-- 默认值 = 4
-- 设置当前会话并行度
SET mview_refresh_dop = 16;
-- 手动刷新
CALL dbms_mview.refresh('mv_orders_items', 'f'); -- 增量
CALL dbms_mview.refresh('mv_orders_items', 'c'); -- 全量
验证结果(从 dba_mvref_run_stats 提取):
|
|
|
|
|
|---|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
结论: mview_refresh_dop 仅对手动调用 DBMS_MVIEW.REFRESH 生效,不影响自动调度任务。
方法 2:DBMS_MVIEW.REFRESH 显式指定 refresh_parallel
CALL dbms_mview.refresh('mv_orders_items', 'f', refresh_parallel => 16);
CALL dbms_mview.refresh('mv_orders_items', 'c', refresh_parallel => 16);
验证结果:
|
|
|
|
|---|---|---|
|
|
|
|
|
|
|
|
结论: 显式指定参数优先级最高,效果与方法 1 一致。
方法 3:设置 MV 的 PARALLEL 属性(部分生效)
ALTER MATERIALIZED VIEW mv_orders_items PARALLEL 16;
CALL dbms_mview.refresh('mv_orders_items', 'f');
CALL dbms_mview.refresh('mv_orders_items', 'c');
验证结果(关键对比数据):
|
|
|
|
|
|---|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
重要结论:
-
ALTER MATERIALIZED VIEW ... PARALLEL N设置的并行度只对自动调度任务生效 -
手动调用 DBMS_MVIEW.REFRESH 不受 MV PARALLEL 属性影响,仍使用 mview_refresh_dop 默认值
并行度优先级总结:
|
|
|
|
|
|---|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
四、刷新数据量变化观测
4.1 数据灌入
-- 维度表:3 行
INSERTINTO items VALUES (1, '笔记本电脑', '电子产品', 5999.00);INSERTINTO items VALUES (2, '办公桌', '家具', 1500.00);INSERTINTO items VALUES (3, '咖啡机', '家电', 899.00);COMMIT;-- 订单表:50000 行(利用 information_schema.tables 笛卡尔积生成序列号)INSERTINTO ordersSELECT seq, MOD(seq, 3) + 1, ROUND(RAND() * 5000, 2),DATE_ADD('2026-06-01', INTERVALMOD(seq, 30) DAY),CASEMOD(seq, 3) WHEN0THEN'SH'WHEN1THEN'BJ'ELSE'GZ'ENDFROM (SELECT @rownum := @rownum + 1AS seqFROM information_schema.tables a, information_schema.tables b, (SELECT @rownum := 0) rLIMIT50000) t;
4.2 查看刷新数据变化量(dba_mvref_change_stats)
SELECT refresh_id, mv_name, tbl_owner, tbl_name,
num_rows_ins, num_rows_upd, num_rows_del, num_rows
FROM oceanbase.dba_mvref_change_stats
WHERE mv_name = 'mv_orders_items'
ORDER BY refresh_id DESC LIMIT 30;
关键输出(数据灌入后的刷新记录):
|
|
|
|
|
|
|
|---|---|---|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
关键发现:
-
refresh_id = 792325 这次刷新捕获到了变更:orders 插入 50000 行,items 插入 3 行 -
前后相邻的刷新记录 num_rows_ins = 0,说明没有数据变更时增量刷新无实际操作
4.3 两表关联查询(run_stats + change_stats)
SELECT r.refresh_id, r.run_owner, r.mviews, r.method,
r.start_time, r.end_time, r.elapsed_time, r.parallelism,
c.tbl_name, c.num_rows_ins, c.num_rows_upd,
c.num_rows_del, c.num_rows
FROM oceanbase.dba_mvref_run_stats r
LEFT JOIN oceanbase.dba_mvref_change_stats c
ON r.refresh_id = c.refresh_id
WHERE r.mviews LIKE '%MV_ORDERS_ITEMS%'
ORDER BY r.start_time DESC LIMIT 30;
此查询将刷新运行统计与基表数据变化量关联,可一次性看到:某次刷新的耗时、并行度、以及各基表的 INSERT/UPDATE/DELETE 行数。
五、基表 DDL 变更限制与处理
5.1 直接修改报错
ALTER TABLE orders MODIFY amount DECIMAL(12,2);
-- ERROR 1235 (0A000): modify column to table with materialized view log is not supported
原因: 基表存在 MLOG,OceanBase 不允许直接修改有 MLOG 依赖的表的列定义。
5.2 正确的处理流程(6 步)
-
停止调度: CALL DBMS_SCHEDULER.DISABLE('MVIEW_REFRESH$J_xxx'); -
删除 MLOG: DROP MATERIALIZED VIEW LOG ON orders;/DROP MATERIALIZED VIEW LOG ON items; -
修改列定义: ALTER TABLE orders MODIFY amount DECIMAL(14,2); -
重建 MLOG: CREATE MATERIALIZED VIEW LOG ON orders WITH PRIMARY KEY, ROWID, SEQUENCE (...) INCLUDING NEW VALUES; -
全量刷新: CALL DBMS_MVIEW.REFRESH('mv_orders_items', 'C');(MLOG 重建后丢失增量记录,必须全量刷新) -
恢复调度: CALL DBMS_SCHEDULER.ENABLE('MVIEW_REFRESH$J_xxx');
六、enable_mlog_auto_maintenance 参数实验
6.1 参数说明
SHOW PARAMETERS LIKE 'enable_mlog_auto_maintenance';
|
|
|
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
6.2 开启后的效果
关键发现: 开启 enable_mlog_auto_maintenance = True 后,创建增量刷新物化视图时不需要手动创建 MLOG,系统会自动管理 MLOG 的创建和维护。
-- 直接创建物化视图(不手动建 MLOG)
CREATEMATERIALIZEDVIEW mv_orders_items
REFRESHFASTONDEMAND
STARTWITHNOW() NEXTNOW() + INTERVAL1MINUTE
ENABLEQUERY REWRITE
ASSELECT ... FROM orders o JOIN items i ON ...;
-- 成功创建,无需预先 CREATE MATERIALIZED VIEW LOG
对比:
-
关闭时:必须先手动 CREATE MATERIALIZED VIEW LOG,再创建 MV -
开启时:系统自动创建和维护 MLOG,简化了操作流程
七、实时物化视图创建与对比
7.1 创建实时物化视图
CREATE MATERIALIZED VIEW mv_orders_items_rt
REFRESH FAST ON DEMAND
ENABLE ON QUERY COMPUTATION -- 关键:启用实时查询计算
AS SELECT ... FROM orders o JOIN items i ON ...;
7.2 对比普通 MV 与实时 MV
SELECT mview_name, refresh_method, rewrite_enabled,
on_query_computation, refresh_dop, staleness
FROM oceanbase.dba_mviews
WHERE mview_name IN ('mv_orders_items', 'mv_orders_items_rt');
|
|
|
|
|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
Y |
|
|
|
0 |
|
|
|
|
关键差异:
-
on_query_computation:实时 MV 为 Y,查询时会实时计算未刷新的增量变更;普通 MV 为 N -
refresh_dop:实时 MV 为 0(由系统自动管理,不可手动指定);普通 MV 可手动设置
八、MLOG 清理机制观测
8.1 实验过程
-- 灌入数据后查看 MLOG
SELECT COUNT(*) FROM mlog$_orders; -- 50000 行
SELECT COUNT(*) FROM mlog$_items; -- 3 行
-- 手动触发增量刷新
CALL DBMS_MVIEW.REFRESH('mv_orders_items', 'F'); -- 1.595s
-- 立即查看 MLOG(刷新后)
SELECT COUNT(*) FROM mlog$_orders; -- 仍然 50000 行!
SELECT COUNT(*) FROM mlog$_items; -- 仍然 3 行!
关键发现: 增量刷新完成后,MLOG 数据没有立即被清理。
8.2 延迟清理
-- 过一段时间后再查
SELECT COUNT(*) FROM mlog$_orders; -- 0 行(已清理)
SELECT COUNT(*) FROM mlog$_items; -- 0 行(已清理)
结论: MLOG 的清理是异步延迟的,不在刷新事务内同步完成。刷新完成后,系统会在后台异步清理已消费的 MLOG 记录。
8.3 刷新统计中的变化量验证
SELECT mv_owner, refresh_id, mv_name, tbl_name,
num_rows_ins, num_rows_upd, num_rows_del, num_rows
FROM oceanbase.dba_mvref_change_stats
WHERE mv_name = 'mv_orders_items'
ORDER BY refresh_id DESC;
关键记录对比:
|
|
|
|
|
|
|---|---|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
分析:
-
num_rows_ins 表示本次刷新实际应用的增量行数 -
num_rows 表示 MLOG 中待处理的总行数 -
当 num_rows_ins = 0 但 num_rows > 0 时(如 refresh_id 298329),说明 MLOG 数据还在但已被标记为已消费,等后台异步清理
九、observer.log 日志观测(全量刷新)
9.1 获取 trace_id
CALL DBMS_MVIEW.REFRESH('mv_orders_items', 'C');
SELECT last_trace_id();
-- YB42C0A8055F-0006559633DE7958-0-0
9.2 observer.log 中的刷新日志
[17:15:53.583] mview complete refresh success(arg={
tenant_id:1014, table_id:500044, parallelism:4,
last_refresh_scn:1783761310230412000,
target_data_sync_scn:1783761352019617000,
use_direct_load_for_complete_refresh:true,
direct_dep_cnt:2,
select_sql:"select ... from (orders as of snapshot ... join items as of snapshot ...)"
}, res={task_id:305871, trace_id:YB42C0A8055F-0006559633DE7958-0-0})
[17:15:53.583] mview refresh finish(ret=0, ret="OB_SUCCESS", param_={
tenant_id:1014, mview_id:500044, refresh_id:305861,
refresh_method:1, retry_id:0, parallel:0,
target_data_sync_scn:1783761352019617000
})
日志字段解读:
|
|
|
|
|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
关键发现:
-
全量刷新使用 as of snapshot 语法读取基表的一致性快照 -
采用 direct load(旁路导入)提升全量刷新性能 -
可通过 trace_id 在 observer.log 中串联完整的刷新执行链路
9.3 刷新统计验证(dba_mvref_stats)
SELECT * FROM oceanbase.DBA_MVREF_STATS WHERE refresh_id=305861;
|
|
|
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
十、核心知识点总结
|
|
|
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
END
瑞蓝创 OceanBase OBCP V4 精英训练营
点击下方图片立即了解详情


▼ 点击「阅读原文」,了解更多产品技术文章


