运营导出的CSV格式不统一,我用pandas统一列名、补全缺失引擎、再输出一张宽表。
code
import pandas as pd
a = pd.read_csv("engine_a.csv")
b = pd.read_csv("engine_b.csv")
a2 = a.rename(columns={"mention_cnt":"mentions_a"})[["day","brand","mentions_a"]]
b2 = b.rename(columns={"cnt":"mentions_b"})[["day","brand","mentions_b"]]
m = a2.merge(b2, on=["day","brand"], how="outer").fillna(0)print(m.sort_values(["day","total"], ascending=[True, False]).head(20))
后续我会加数据质量检查:单日突增超过3倍sigma就报警,防止抓取解析bug造成假阳性。
pandas在大表上慢的话,我会换polars。企业AI落地数据量目前还行。