专题追踪：大语言模型专题第2期- 大数跨境

首页

专题追踪：大语言模型专题第2期

营在社媒

2025-05-30

导读：在消费者行为日益复杂、数据量激增的数字化时代，企业面临市场研究成本高、周期长的核心痛点。传统调研方法难以捕捉瞬息万变的消费者感知，而大语言模型的崛起正带来范式变革。

点击上方蓝字关注，了解更多前沿研究

在消费者行为日益复杂、数据量激增的数字化时代，企业面临市场研究成本高、周期长的核心痛点。传统调研方法难以捕捉瞬息万变的消费者感知，而大语言模型（LLM）的崛起正带来范式变革——这些“数字大脑”不仅能实时解析海量文本，更能模拟人类认知过程，为市场研究注入前所未有的效率与深度。

本期聚焦《Marketing Science》等期刊2024年-2025年的三项研究，揭示LLM在消费者感知分析、偏好预测、内容优化中的革命性应用，探索人机协同的营销新时代。

1. Determining the Validity of Large Language Models for Automated Perceptual Analysis

自动化感知分析中大型语言模型的有效性验证
期刊：Marketing Science
作者：Li, PY; Castelo, N; Katona, Z; Sarvary, M
日期：2024年3月-4月

英文摘要：
This paper explores the potential of large language models (LLMs) to substitute for human participants in market research. Such LLMs can be used to generate text given a prompt. We argue that perceptual analysis is a particularly promising use case for such automated market research for certain product categories. The proposed new method generates outputs that closely match those generated from human surveys: agreement rates between human- and LLM- generated data sets reach over 75%. Moreover, this applies for perceptual analysis based on both brand similarity measures and product attribute ratings. The paper demonstrates that, for some categories, this new method of fully or partially automated market research will increase the efficiency of market research by meaningfully speeding up the process and potentially reducing the cost. Further results also suggest that with an ever larger training corpus applied to large language models, LLM-based market research will be applicable to answer more nuanced questions based on demographic variables or contextual variation that would be prohibitively expensive or infeasible with human respondents.

中文摘要：
本文探索了大语言模型（LLM）替代人类参与者进行市场研究的潜力。这类模型能够根据提示生成文本。我们认为，对于特定产品类别，感知分析是自动化市场研究中极具前景的应用场景。所提出的新方法生成的输出与人类调查结果高度吻合：人类数据与LLM生成数据的一致性率超过75%。该结果适用于基于品牌相似度和产品属性评级的感知分析。研究表明，对于某些品类，这种全自动或半自动的新方法将显著加速研究进程并降低成本，从而提升市场研究效率。进一步结果还表明，随着训练语料库的扩大，基于LLM的市场研究将能回答更精细的问题（如基于人口统计变量或情境变化），而此类问题若用人类受访者则成本过高或难以实现。

2. Can Large Language Models Capture Human Preferences?

大语言模型能否捕捉人类偏好？
期刊：Marketing Science
作者：Goli, A; Singh, A
日期：2024年7月-8月

英文摘要：
We explore the viability of large language models (LLMs), specifically OpenAI's GPT-3.5 and GPT-4, in emulating human survey respondents and eliciting preferences, with a focus on intertemporal choices. Leveraging the extensive literature on intertemporal discounting for benchmarking, we examine responses from LLMs across various languages and compare them with human responses, exploring preferences between smaller, sooner and larger, later rewards. Our findings reveal that both generative pretrained transformer (GPT) models demonstrate less patience than humans, with GPT-3.5 exhibiting a lexicographic preference for earlier rewards unlike human decision makers. Although GPT-4 does not display lexicographic preferences, its measured discount rates are still considerably larger than those found in humans. Interestingly, GPT models show greater patience in languages with weak future tense references, such as German and Mandarin, aligning with the existing literature that suggests a correlation between language structure and intertemporal preferences. We demonstrate how prompting GPT to explain its decisions, a procedure we term "chain-of-thought conjoint," can mitigate, but does not eliminate, discrepancies between LLM and human responses. Although directly eliciting preferences using LLMs may yield misleading results, combining chain-of-thought conjoint with topic modeling aids in hypothesis generation, enabling researchers to explore the underpinnings of preferences. Chain-of-thought conjoint provides a structured framework for marketers to use LLMs to identify potential attributes or factors that can explain preference heterogeneity across different customers and contexts.

中文摘要：
我们探索了大型语言模型（LLM），特别是OpenAI的GPT-3.5和GPT-4，在模拟人类调查对象和获取偏好方面的可行性，重点关注跨期选择。利用跨期折现的广泛文献作为基准，我们检验了LLM在不同语言中的响应，并将其与人类响应进行比较，探讨了在较小即时奖励与较大延迟奖励之间的偏好。研究发现，两种GPT模型均表现出比人类更低的耐心程度：GPT-3.5对即时奖励呈现词典编撰式偏好，与人类决策者不同；GPT-4虽无此现象，但其折现率仍显著高于人类。有趣的是，GPT模型在弱未来时态引用语言（如德语、汉语）中表现出更高耐心，这与语言结构和跨期偏好存在相关性的现有文献一致。我们提出的“思维链联合分析”方法（通过提示GPT解释决策）可缩小LLM与人类响应的差异，但无法完全消除。尽管直接使用LLM获取偏好可能产生误导性结果，但思维链联合分析与主题建模的结合能辅助假设生成，帮助研究者探索偏好形成的底层机制。该方法为营销人员提供了结构化框架，以识别解释不同客户和情境下偏好异质性的关键属性。

3. LOLA: LLM-Assisted Online Learning Algorithm for Content Experiments

LOLA：基于LLM的内容实验在线学习算法
期刊：Marketing Science
作者：Ye, ZK; Yoganarasimhan, H; Zheng, YF
日期：2025年3月

英文摘要：
Modern media firms require automated and efficient methods to identify content that is most engaging and appealing to users. Leveraging a large-scale data set from Upworthy (a news publisher), which includes 17,681 headline A/B tests, we first investigate the ability of three pure-large language model (LLM) approaches to identify the catchiest headline: prompt-based methods, embedding-based methods, and fine-tuned open-source LLMs. Prompt-based approaches perform poorly, while both OpenAI embedding-based models and the fine-tuned Llama-3-8B achieve marginally higher accuracy than random predictions. In sum, none of the pure LLM-based methods can predict the best-performing headline with high accuracy. We then introduce the LLM-assisted online learning algorithm (LOLA), a novel framework that integrates LLMs with adaptive experimentation to optimize content delivery. LOLA combines the best pure-LLM approach with the upper confidence bound algorithm to allocate traffic and maximize clicks adaptively. Our numerical experiments on Upworthy data show that LOLA outperforms the standard A/B test method (the current status quo at Upworthy), pure bandit algorithms, and pure-LLM approaches, particularly in scenarios with limited experimental traffic. Our approach is scalable and applicable to content experiments across various settings where firms seek to optimize user engagement, including digital advertising and social media recommendations.

中文摘要：
现代媒体公司需要自动化高效的方法识别最具吸引力的内容。基于Upworthy（新闻出版商）包含17,681次标题A/B测试的大规模数据集，我们首先检验了三种纯LLM方法识别最佳标题的能力：提示工程法、嵌入模型法和微调开源LLM法。提示工程法表现不佳，而基于OpenAI嵌入的模型和微调的Llama-3-8B仅略优于随机预测。总体而言，纯LLM方法均无法高精度预测最佳标题。为此我们提出LOLA框架（LLM辅助在线学习算法），将LLM与自适应实验结合以优化内容投放。LOLA融合最优纯LLM方法与置信上界算法，自适应分配流量并最大化点击量。在Upworthy数据上的实验表明，LOLA显著优于标准A/B测试、纯赌博算法及纯LLM方法，尤其在实验流量有限的场景下。该框架具有可扩展性，适用于数字广告、社交媒体推荐等需优化用户参与度的场景。