![[行空板+大模型]智能家居助手——GPT3.5 function calling控制硬件](https://cdn.10100.com/user/3f833159ef0c8fcc90a85a496e106814.png?x-oss-process=style/180x)
点击 蘑菇云创造，关注我们！
案例一：“行空板+大模型”——基于ChatGLM的多角色交互式聊天机器人
一、导语（项目功能介绍）:
OpenAI在2023年6月的更新中为Chat Completions 模型增加了function calling功能。我们知道通过ChatCompletion，我们能实现通过调用API完成与ChatGPT的对话。而有了Funtion Calling功能,可以让模型不仅仅能根据自身预训练的数据库进行知识问答，还能额外挂载一个函数库，根据用户的提问去函数库检索，按照用户的实际需求去调用外部函数并获取函数的运行结果。
可以说funtion calling彻底的改变了开发者与模型互动的方式。这个功能允许开发者描述函数给AI模型，然后模型可以智能地决定输出一个包含调用这些函数的参数的JSON对象。简单来说，大模型的function calling功能允许我们使用自然语言的形式调用函数，实现让GPT模型调用谷歌邮箱API（Gmail API），自动让GPT模型读取邮件，并自动进行回复等等 。整理了目前支持funtion calling的模型，如下图。
借助funtion calling功能，用户能通过简单的语句实现控制硬件，而无需深入的研究底层的代码逻辑。想象一下，假设现在你是一个对python一无所知的用户，但是你对行空板和硬件产品都很感兴趣，现在你只需要将硬件正确的接在对应的引脚位置，然后告诉大模型你想实现的功能，比如“我在21号引脚接了一个风扇，我想要打开风扇”，模型就能帮助你调用相关的函数控制风扇的启动，大模型的funtion calling功能完全能帮助我们实现这一场景。
举个例子，现在我在定义了一个控制行空板板载蜂鸣器播放音调的函数play_music，并且告诉了模型这个函数的功能。那么可以有以下对话场景的出现。
整个实现的逻辑如下图：
本篇帖子就来展示如何实现这一功能。本文旨在提供方法和思路，笔者希望通过这篇文章抛砖引玉，展示给大家看通过大模型控制硬件的可能性，以行空板板载蜂鸣器、灯带、风扇模拟生活中智能家居的声、光、动。也希望大家在看完这篇帖子后能发挥创意，实现行空板结合大模型控制硬件的更多好玩的应用。（PS：本项目的实现需借助魔法上网。）
二、演示视频
三、软硬件
四、funtion calling 的实现流程
智谱AI在官方文档里给出了使用模型进行工具调用的流程，ChatGLM3-6b和GPT3.5的实现逻辑是一样的，因此我们可以进行参考。https://github.com/THUDM/ChatGLM3/blob/main/tools_using_demo/README.md
首先，我们要定义好函数工具，比如控制行空板板载蜂鸣器播放音乐的函数工具play_music、用于控制风扇硬件的函数工具fan_action、用于控制LED灯带的函数工具led_light_action
接着我们要定义一个工具列表，向模型描述这些函数工具的名称、函数的具体功能、调用函数所需的参数、参数的具体类型、参数的详细信息。这是官方给的样例，按照样例能获得最好的性能。
tools = [{"name": "track","description": "追踪指定股票的实时价格","parameters": {"type": "object","properties": {"symbol": {"description": "需要追踪的股票代码"}},"required": ['symbol']}},{"name": "text-to-speech","description": "将文本转换为语音","parameters": {"type": "object","properties": {"text": {"description": "需要转换成语音的文本"},"voice": {"description": "要使用的语音类型（男声、女声等）"},"speed": {"description": "语音的速度（快、中等、慢等）"}},"required": ['text']}}]system_info = {"role": "system", "content": "Answer the following questions as best as you can. You have access to the following tools:", "tools": tools}
五、准备工作
1、申请Open AI API key：https://platform.openai.com/api-keys
2、申请微软语音服务API：
https://portal.azure.com/#view/Microsoft_Azure_ProjectOxford/CognitiveServicesHub/~/SpeechServices
3.行空板联网
调用api必须要联网。由于在本项目中，我们使用了OpenAI的api和微软的语音api，所以我们要为行空板连接网络。（1）打开浏览器，输入“10.1.2.3”进行行空板页面。（2）选择“网络设置”,选择WIFI ，输入密码，注意行空板仅支持2.4GWIFI热点。点击“连接”，行空板成功联网会显示“连接成功”。
六、tool描述和funtion实现
接下来，我将按照funtion calling的实现过程，按照“编写控制硬件的函数工具——定义工具列表描述函数工具的详细信息——解析用户输入传给模型 ”进行代码的编写。
6.1蜂鸣器6.1.1pinpong库示例程序
为了完整的实现使用funtion calling功能控制行空板的板载蜂鸣器，我们先一起来看看pinpong库中控制蜂鸣器播放音调的示例代码是怎么写的。pinpong库有关蜂鸣器的示例程序如下。
import timefrom pinpong.board import Board,Pinfrom pinpong.extension.unihiker import *Board().begin()#初始化#音乐 DADADADUM ENTERTAINER PRELUDE ODE NYAN RINGTONE FUNK BLUES BIRTHDAY WEDDING FUNERAL PUNCHLINE#音乐 BADDY CHASE BA_DING WAWAWAWAA JUMP_UP JUMP_DOWN POWER_UP POWER_DOWN#播放模式 Once(播放一次) Forever(一直播放) OnceInBackground(后台播放一次) ForeverInBackground(后台一直播放)buzzer.play(buzzer.DADADADUM, buzzer.Once) #播放音乐一次while True:time.sleep(1) #等待1秒 保持状态
6.1.2定义控制蜂鸣器播放音乐的函数play_music
根据pinpong库中控制行空板板载蜂鸣器的示例程序，我们可以写出一个play_music函数来控制蜂鸣器播放内置音乐。它接受一个参数“music”，用于指定要播放的音乐名称。if music in [...]检查传入的音乐名称是否在预定义的音乐列表中。如果在，使用buzzer.play函数播放音乐，如果不在就提醒示没有找到指定的音乐名称，无法进行播放。
def play_music(music):if music in ["DADADADUM", "ENTERTAINER", "PRELUDE", "ODE", "NYAN", "RINGTONE", "FUNK", "BLUES", "BIRTHDAY", "WEDDING", "FUNERAL", "PUNCHLINE", "BADDY", "CHASE", "BA_DING", "WAWAWAWAA", "JUMP_UP", "JUMP_DOWN", "POWER_UP", "POWER_DOWN"]:buzzer.play(getattr(buzzer, music), buzzer.OnceInBackground)print(f"已调用，播放{music}")else:print("对不起，我这里没有这个音乐。")
6.1.3定义工具列表
按照官方样例的格式，详细的描述play_music函数的名称、描述以及参数。
tools = [{"type": "function","function": {"name": "play_music","description": "Plays music in the house when the user is bored or needs relaxation. The music list includes 'DADADADUM', 'ENTERTAINER', 'PRELUDE', 'ODE', 'NYAN', 'RINGTONE', 'FUNK', 'BLUES', 'BIRTHDAY', 'WEDDING', 'FUNERAL', 'PUNCHLINE', 'BADDY', 'CHASE', 'BA_DING', 'WAWAWAWAA', 'JUMP_UP', 'JUMP_DOWN', 'POWER_UP', 'POWER_DOWN'.","parameters": {"type": "object","properties": {"music": {"type": "string","enum": ["DADADADUM", "ENTERTAINER", "PRELUDE", "ODE", "NYAN", "RINGTONE", "FUNK", "BLUES", "BIRTHDAY", "WEDDING", "FUNERAL", "PUNCHLINE", "BADDY", "CHASE", "BA_DING", "WAWAWAWAA", "JUMP_UP", "JUMP_DOWN", "POWER_UP", "POWER_DOWN"],"description": "The music to play in the house."},},"required": ["music"],},}}]
接着，定义一个函数，使用chatcompletion向模型发送聊天完成请求
def chat_completion_request(messages, tools=None, tool_choice=None):"""发送 Chat Completion 请求给 OpenAI GPT-3.5-turbo 模型，并返回模型的响应。参数：- messages: 包含对话历史的列表，每个对话历史是一个字典，包含 "role" 和 "content"。- tools: （可选）包含工具描述的列表，每个工具是一个字典。- tool_choice: （可选）工具选择，用于指定使用哪个工具。返回：- response: 发送请求后的模型响应。"""headers = {"Content-Type": "application/json","Authorization": "Bearer " + OPENAI_API_KEY,}# 构建请求的 JSON 数据json_data = {"model": "gpt-3.5-turbo", "messages": messages}if tools is not None:json_data.update({"tools": tools})if tool_choice is not None:json_data.update({"tool_choice": tool_choice})try:# 发送 POST 请求到 OpenAI APIresponse = requests.post("https://api.openai.com/v1/chat/completions",headers=headers,json=json_data,)return responseexcept Exception as e:# 处理异常情况print("Unable to generate ChatCompletion response")print(f"Exception: {e}")return e
接下来，就是在主循环中，获取用户输入和模型的响应，与一般的打印模型的响应不同，这里要多加一项即检查模型的响应中是否含有工具调用的相关信息(tool_calls)。如果存在，提取函数和参数信息，然后调用相应的函数执行操作。
while True:# 用户输入，通过命令行获取用户的自然语言输入user_input = input('请输入：')# 将用户输入添加到对话历史中，定义用户的角色为 "user"conversation.append({"role": "user", "content": user_input})# 调用 chat_completion_request 函数，向模型发送对话历史，并获取模型的响应response = chat_completion_request(conversation, tools)print("Model Response: ", response.json())# 提取助手的回复内容assistant_response = response.json()["choices"][0]["message"]["content"]# 如果助手的回复为空，将其设置为默认值 "执行操作"if assistant_response is None:assistant_response = "执行操作"# 打印助手的回复print(f"Assistant: {assistant_response}")# 检查模型的响应中是否包含 tool_calls（Function Calling 操作）if 'tool_calls' in response.json()["choices"][0]["message"]:# 提取工具调用信息中的函数和参数function_call = response.json()["choices"][0]["message"]["tool_calls"][0]["function"]arguments = json.loads(function_call["arguments"])# 调用 play_music 函数，根据模型的指令执行音乐播放操作play_music(arguments["music"])# 将助手的回复添加到对话历史中，定义助手的角色为 "assistant"conversation.append({"role": "assistant", "content": assistant_response})
6.1.4使用funtion calling功能控制行空板蜂鸣器的完整程序如下
import requestsimport jsonfrom pinpong.board import Board, Pinfrom pinpong.extension.unihiker import *import openaiBoard().begin() # 初始化OPENAI_API_KEY=' ' # OpenAI API 密钥def chat_completion_request(messages, tools=None, tool_choice=None):headers = {"Content-Type": "application/json","Authorization": "Bearer " + OPENAI_API_KEY,}json_data = {"model": "gpt-3.5-turbo", "messages": messages}if tools is not None:json_data.update({"tools": tools})if tool_choice is not None:json_data.update({"tool_choice": tool_choice})try:response = requests.post("https://api.openai.com/v1/chat/completions",headers=headers,json=json_data,)return responseexcept Exception as e:print("Unable to generate ChatCompletion response")print(f"Exception: {e}")return etools = [{"type": "function","function": {"name": "play_music","description": "Plays music in the house when the user is bored or needs relaxation. The music list includes 'DADADADUM', 'ENTERTAINER', 'PRELUDE', 'ODE', 'NYAN', 'RINGTONE', 'FUNK', 'BLUES', 'BIRTHDAY', 'WEDDING', 'FUNERAL', 'PUNCHLINE', 'BADDY', 'CHASE', 'BA_DING', 'WAWAWAWAA', 'JUMP_UP', 'JUMP_DOWN', 'POWER_UP', 'POWER_DOWN'.","parameters": {"type": "object","properties": {"music": {"type": "string","enum": ["DADADADUM", "ENTERTAINER", "PRELUDE", "ODE", "NYAN", "RINGTONE", "FUNK", "BLUES", "BIRTHDAY", "WEDDING", "FUNERAL", "PUNCHLINE", "BADDY", "CHASE", "BA_DING", "WAWAWAWAA", "JUMP_UP", "JUMP_DOWN", "POWER_UP", "POWER_DOWN"],"description": "The music to play in the house."},},"required": ["music"],},}}]def play_music(music):if music in ["DADADADUM", "ENTERTAINER", "PRELUDE", "ODE", "NYAN", "RINGTONE", "FUNK", "BLUES", "BIRTHDAY", "WEDDING", "FUNERAL", "PUNCHLINE", "BADDY", "CHASE", "BA_DING", "WAWAWAWAA", "JUMP_UP", "JUMP_DOWN", "POWER_UP", "POWER_DOWN"]:buzzer.play(getattr(buzzer, music), buzzer.OnceInBackground)print(f"已调用，播放{music}")else:print("对不起，我这里没有这个音乐。")conversation = [{"role": "system", "content": "You are a smart home assistant that can play music for relaxation when the user is bored or needs it. You can suggest playing music when the user seems bored or tired."}]while True:user_input = input('请输入：')conversation.append({"role": "user", "content": user_input})response = chat_completion_request(conversation, tools)print("Model Response: ", response.json())assistant_response = response.json()["choices"][0]["message"]["content"]if assistant_response is None:assistant_response = "执行操作"print(f"Assistant: {assistant_response}")if 'tool_calls' in response.json()["choices"][0]["message"]:function_call = response.json()["choices"][0]["message"]["tool_calls"][0]["function"]arguments = json.loads(function_call["arguments"])play_music(arguments["music"])conversation.append({"role": "assistant", "content": assistant_response})
6.2风扇
在生活中，我们使用风扇时能通过调节档位来改变风扇的转速。这里我们能用PWM模拟输出实现这一功能。
6.2.1pinpong库关于PWM输出控制好风扇的示例程序
关于行空板支持PWM的引脚号，可以查看官方的说明：PWM（模拟输出：https://www.unihiker.com.cn/wiki/pinpong_python_lib#5.4-%E6%A8%A1%E6%8B%9F%E8%BE%93%E5%87%BA%EF%BC%88PWM%EF%BC%89。
根据官方对PWM的说明，我们使用PWM控制风扇的代码，如下。
from pinpong.board import Board,PinBoard(" ").begin()fan = Pin(Pin.P22, Pin.PWM)fan.write_analog(800)
6.2.2定义控制风扇的函数
通过示例程序，我们可以看出通过funtion calling 使模型控制风扇，首先我们要告诉模型：风扇的引脚位置的初始化 。接着我们要告诉模型对风扇的操作，包括打开风扇、提高转速、降低转速、关闭风扇（通过PWM输出控制转速）。
我们依旧是先写函数，根据函数完成工具的描述和工具调用的代码。
我们先来完成风扇初始化的代码。站在用户的角度，使用风扇时，会将使用引脚线将风扇接在行空板能使用PWM的引脚的位置，然后将引脚的位置告诉模型。为了让模型知道引脚的位置，我们要先定义一个能支持PWM的引脚的列表。
定义一个全局变量fan,用于存储风扇的引脚信息。
pin_map = {21: Pin.P21,22: Pin.P22,23: Pin.P23,}
接着，定义控制风扇的函数fan_action,通过分支结构判断指定的风扇的类型，通过用户的输入完成对风扇的初始化和控制等。fan_action有三个参数，‘action’:指定要执行的动作，用于使用分支结构控制风扇的状态。'pn_number'用于风扇初始化时确定风扇连接的引脚号。‘user_input’用于确定用户对风扇的控制指令。
值得注意的是，当用户没有进行初始化就说打开风扇或者说了错误的引脚号时，要打印错误信息，提醒用户给出正确的引脚位置。
def fan_action(action, pin_number=None, user_input=None):"""控制风扇的行为。参数：- action: 指定要执行的动作，可以是 "initialize"（初始化）或其他操作。- pin_number: （仅在 action 为 "initialize" 时需要）风扇连接的引脚号。- user_input: （仅在执行其他操作时需要）用户的输入，用于确定具体的操作。返回：无返回值。通过打印信息告知用户操作的结果。详细解释：1. 使用全局变量 `fan`，用于存储风扇的引脚信息。2. 如果 action 是 "initialize"，则进行初始化操作。a. 检查传入的引脚号是否在 pin_map 中，如果不在则打印错误信息。b. 使用 Pin 类创建风扇对象，并设置为 PWM 模式。c. 打印初始化成功的信息。3. 如果 action 不是 "initialize"，执行相应的操作。a. 检查风扇是否已经初始化，如果未初始化则打印错误信息。b. 根据用户输入执行相应的操作：- "turn on": 开启风扇，设置 PWM 为 800。- "turn off": 关闭风扇，设置 PWM 为 0。- "increase": 增加风扇转速，设置 PWM 为 1023。- "decrease": 减少风扇转速，设置 PWM 为 512。c. 打印执行操作成功的信息。使用示例：```# 初始化风扇，连接到引脚号 5fan_action("initialize", pin_number=5)# 执行操作，根据用户输入控制风扇fan_action("operate", user_input="turn on")```注意：确保在使用之前设置全局变量 pin_map 以包含正确的引脚映射。"""global fan# 如果是初始化操作if action == "initialize":if pin_number not in pin_map:print(f"错误的引脚号：{pin_number}")return# 使用 Pin 类创建风扇对象，并设置为 PWM 模式fan = Pin(pin_map[pin_number], Pin.PWM)print(f"风扇已初始化，接在{pin_number}号端口")else:# 如果风扇未初始化if fan is None:print("对不起，风扇尚未初始化，请先告诉我引脚号")return# 根据用户输入执行相应的操作if "turn on" in user_input:fan.write_analog(800)print("风扇已开启")elif "turn off" in user_input:fan.write_analog(0)print("风扇已关闭")elif "increase" in user_input:fan.write_analog(1023)print("风扇转速已增加")elif "decrease" in user_input:fan.write_analog(512)print("风扇转速已减少")
6.2.3添加对风扇函数的工具描述
详细的解释函数以及各个参数的作用。
tools = [{"type": "function","function": {"name": "fan_action","description": "This function initializes or controls the fan based on the given action and user's command. It can initialize the fan, turn on the fan, turn off the fan, increase the fan speed, or decrease the fan speed. It will print an error message if the fan has not been initialized.","parameters": {"type": "object","properties": {"action": {"type": "string","enum": ["initialize", "turn_on", "turn_off", "increase_speed", "decrease_speed"],"description": "The action to perform on the fan.",},"pin_number": {"type": "integer","description": "The pin number where the fan is connected. Required for 'initialize' action.",},"user_input": {"type": "string","description": "The user's command to control the fan. Required for 'turn on', 'turn off', 'increase speed', 'decrease speed' actions.",},},"required": ["action"],},},},]
6.2.4使用funtion calling功能控制风扇的完整程序如下
# 导入所需的库和模块import requestsimport jsonfrom pinpong.board import Board, Pinimport openaiimport time# 初始化 Pinpong BoardBoard(" ").begin()# 设置你的 OpenAI API 密钥OPENAI_API_KEY = ' '# 初始化风扇变量fan = None# 引脚号映射到 Pinpong Pinspin_map = {21: Pin.P21,22: Pin.P22,23: Pin.P23,24: Pin.P24,}# 发起聊天请求到 OpenAI API 的函数def chat_completion_request(messages, tools=None, tool_choice=None):headers = {"Content-Type": "application/json","Authorization": "Bearer " + OPENAI_API_KEY,}json_data = {"model": "gpt-3.5-turbo-1106", "messages": messages}if tools is not None:json_data.update({"tools": tools})if tool_choice is not None:json_data.update({"tool_choice": tool_choice})try:response = requests.post("https://api.openai.com/v1/chat/completions",headers=headers,json=json_data,)return responseexcept Exception as e:print("无法生成 ChatCompletion 响应")print(f"异常: {e}")return e# 定义风扇控制工具tools = [{"type": "function","function": {"name": "fan_action","description": "此函数根据给定的操作和用户的命令初始化或控制风扇。可以初始化风扇、打开风扇、关闭风扇、增加风扇速度或减少风扇速度。如果风扇未初始化，将打印错误消息。","parameters": {"type": "object","properties": {"action": {"type": "string","enum": ["initialize", "turn_on", "turn_off", "increase_speed", "decrease_speed"],"description": "在风扇上执行的操作。",},"pin_number": {"type": "integer","description": "风扇连接的引脚编号。对于 'initialize' 操作，此项为必填。",},"user_input": {"type": "string","description": "用户控制风扇的命令。对于 'turn on'、'turn off'、'increase speed'、'decrease speed' 操作，此项为必填。",},},"required": ["action"],},},},]# 执行基于用户命令的风扇操作的函数def fan_action(action, pin_number=None, user_input=None):global fanif action == "initialize":if pin_number not in pin_map:print(f"错误的引脚号：{pin_number}")returnfan = Pin(pin_map[pin_number], Pin.PWM)print(f"风扇已初始化，接在{pin_number}号端口")else:if fan is None:print("风扇尚未初始化")returnif "turn on" in user_input:fan.write_analog(800)print("风扇已开启")elif "turn off" in user_input:fan.write_analog(0)print("风扇已关闭")elif "increase" in user_input:fan.write_analog(1023)print("风扇转速已增加")elif "decrease" in user_input:fan.write_analog(512)print("风扇转速已减少")# 系统发起的对话消息conversation = [{"role": "system", "content": "你是一个能够控制风扇的有用助手。"},]# 主循环while True:user_input = input('请输入：')conversation.append({"role": "user", "content": user_input})# 向 OpenAI API 发起聊天请求response = chat_completion_request(conversation, tools)print("模型响应: ", response.json())# 处理模型的响应并执行请求的函数if 'tool_calls' in response.json()["choices"][0]["message"]:function_call = response.json()["choices"][0]["message"]["tool_calls"][0]["function"]print(f"模型调用的函数: {function_call['name']}")if function_call["name"] == "fan_action":arguments = json.loads(function_call["arguments"])fan_action(arguments["action"], arguments.get("pin_number"), arguments.get("user_input"))# 显示助手的响应assistant_response = response.json()["choices"][0]["message"]["content"]if assistant_response is None:assistant_response = "执行操作"print(f"助手: {assistant_response}")# 将助手的响应添加到对话中conversation.append({"role": "assistant", "content": assistant_response})
6.3灯带6.3.1pinpong库灯带示例程序
通过pinpong库的示例程序可以看出，灯带的初始化需要两个参数：（1）灯带的引脚位置 （2）灯珠的数量。而灯珠颜色的设置是通过修改RGB值来实现的。
# -*- coding: utf-8 -*-import timefrom pinpong.board import Board,Pin,NeoPixelNEOPIXEL_PIN = Pin.D7 #灯的引脚号PIXELS_NUM = 4 #灯珠数量数np = NeoPixel(Pin(NEOPIXEL_PIN), PIXELS_NUM) #灯带的初始化while True:np[0] = (0, 255 ,0) #设置第一个灯RGB亮度np[1] = (255, 0, 0) #设置第二个灯RGB亮度np[2] = (0, 0, 255) #设置第三个灯RGB亮度np[3] = (255, 0, 255) #设置第四个灯RGB亮度time.sleep(1)
6.3.2定义点亮灯带的函数
在定义点亮灯带的函数时，我们先来思考用户会怎样使用灯带。当用户将灯带接在行空板的引脚上后，首先会告诉模型灯带的引脚号和灯珠的数量，只有这样，才能完成灯带的正确的初始化。因此，我们需要先定义两个全局变量，用于灯带的初始化操作
led = None #用于初始化灯带对象num_beads_global = None #用于初始化灯珠数量
当灯带初始化完成之后，用户会告诉模型想点亮的灯珠的数量和颜色，如“我想点亮4颗灯珠为红色”，这时，我们需要将红色转换成其对应的RGB值。为了实现这样的功能，我们可以定义一个颜色字典，将颜色名字与其RGB值对应起来。
# 创建颜色字典COLOR_DICT = {"red": [255, 0, 0],"green": [0, 255, 0],"blue": [0, 0, 255],"white": [255, 255, 255],"black": [0, 0, 0],"yellow": [255, 255, 0],"pink": [255, 105, 180],"purple": [128, 0, 128]}
接下来，定义函数 led_light_action ，用于初始化和控制 LED 灯带，支持两种操作：initialize（初始化）和 lightup（点亮）。在 initialize 操作中，它根据给定的引脚号和灯珠数量初始化 LED 灯带，而在 lightup 操作中，它点亮指定数量的 LED 灯珠，并设置它们的颜色。同时，函数会进行错误检查，确保必要的参数被提供，并在出现错误时输出相应的错误消息。
def led_light_action(action, pin_number=None, num_lights=None, color=None):global ledglobal num_beads_global# 如果动作是初始化if action == "initialize":# 检查参数是否完整if pin_number is None or num_lights is None:print("错误：'pin_number' 和 'num_lights' 在 'initialize' 动作中是必需的。")return# 初始化 LED 灯led = NeoPixel(Pin(pin_number), num_lights)num_beads_global = num_lightsprint(f"LED 灯已在引脚 {pin_number} 上用 {num_lights} 颗珠子进行初始化。")# 如果动作是点亮elif action == "lightup":# 检查 LED 是否已初始化if led is None:print("错误：LED 灯尚未初始化。")return# 检查参数是否完整if num_lights is None or color is None:print("错误：'num_lights' 和 'color' 在 'lightup' 动作中是必需的。")return# 检查 'num_lights' 是否为正整数，且不大于全局珠子数量if not isinstance(num_lights, int) or num_lights <= 0 or num_lights > num_beads_global:print("错误：'num_lights' 必须是正整数且不能大于珠子数量。")return# 获取颜色对应的 RGB 值color_rgb = COLOR_DICT.get(color.lower())if color_rgb is None:print(f"错误：未知颜色 '{color}'")return# 逐个点亮指定数量的珠子for i in range(num_lights):led[i] = tuple(color_rgb)led.write(i, color_rgb[0], color_rgb[1], color_rgb[2])print(f"点亮了 {num_lights} 颗珠子，颜色为 {color}。")# 如果动作未知else:print(f"错误：未知动作 '{action}'")
6.3.3点亮灯带的函数的工具描述
tools = [{"type": "function","function": {"name": "led_light_action","description": "This function performs actions on a LED light. It can initialize the LED light on a specific pin with a certain number of beads, or light up a certain number of beads in a specific color.","parameters": {"type": "object","properties": {"action": {"type": "string","enum": ["initialize", "lightup"],"description": "The action to perform. 'initialize' will set up the LED light on a specific pin with a certain number of beads. 'lightup' will light up a certain number of beads in a specific color."},"pin_number": {"type": "integer","description": "The pin number where the LED light is connected. This is required when the 'action' is 'initialize'."},"num_lights": {"type": "integer","description": "The number of beads to light up or initialize. This is required when the 'action' is 'initialize' or 'lightup'."},"color": {"type": "string","description": "The color to use when lighting up the beads. This is required when the 'action' is 'lightup'."},},"required": ["action"],},},}]
6.3.4使用funtion calling 功能点亮灯带的完整程序如下
import requestsimport jsonfrom pinpong.board import Board, Pin, NeoPixelBoard().begin() # 初始化OPENAI_API_KEY = ' ' # OpenAI API 密钥# 创建颜色字典COLOR_DICT = {"red": [255, 0, 0],"green": [0, 255, 0],"blue": [0, 0, 255],"white": [255, 255, 255],"black": [0, 0, 0],"yellow": [255, 255, 0],"pink": [255, 105, 180],"purple": [128, 0, 128]}led = Nonenum_beads_global = Nonedef chat_completion_request(messages, tools=None, tool_choice=None):headers = {"Content-Type": "application/json","Authorization": "Bearer " + OPENAI_API_KEY,}json_data = {"model": "gpt-3.5-turbo", "messages": messages}if tools is not None:json_data.update({"tools": tools})if tool_choice is not None:json_data.update({"tool_choice": tool_choice})try:response = requests.post("https://api.openai.com/v1/chat/completions",headers=headers,json=json_data,)return responseexcept Exception as e:print("Unable to generate ChatCompletion response")print(f"Exception: {e}")return Nonedef led_light_action(action, pin_number=None, num_lights=None, color=None):global ledglobal num_beads_globalif action == "initialize":if pin_number is None or num_lights is None:print("Error: 'pin_number' and 'num_lights' are required for 'initialize' action.")returnled = NeoPixel(Pin(pin_number), num_lights)num_beads_global = num_lightsprint(f"LED light initialized on pin {pin_number} with {num_lights} beads.")elif action == "lightup":if led is None:print("Error: LED light has not been initialized.")returnif num_lights is None or color is None:print("Error: 'num_lights' and 'color' are required for 'lightup' action.")returnif not isinstance(num_lights, int) or num_lights <= 0 or num_lights > num_beads_global:print("Error: 'num_lights' must be a positive integer and not greater than the number of beads.")returncolor_rgb = COLOR_DICT.get(color.lower())if color_rgb is None:print(f"Error: Unknown color '{color}'")returnfor i in range(num_lights):led[i] = tuple(color_rgb)led.write(i, color_rgb[0], color_rgb[1], color_rgb[2])print(f"Lit up {num_lights} beads with color {color}.")else:print(f"Error: Unknown action '{action}'")tools = [{"type": "function","function": {"name": "led_light_action","description": "This function performs actions on a LED light. It can initialize the LED light on a specific pin with a certain number of beads, or light up a certain number of beads in a specific color.","parameters": {"type": "object","properties": {"action": {"type": "string","enum": ["initialize", "lightup"],"description": "The action to perform. 'initialize' will set up the LED light on a specific pin with a certain number of beads. 'lightup' will light up a certain number of beads in a specific color."},"pin_number": {"type": "integer","description": "The pin number where the LED light is connected. This is required when the 'action' is 'initialize'."},"num_lights": {"type": "integer","description": "The number of beads to light up or initialize. This is required when the 'action' is 'initialize' or 'lightup'."},"color": {"type": "string","description": "The color to use when lighting up the beads. This is required when the 'action' is 'lightup'."},},"required": ["action"],},},}]messages = [{"role": "system", "content": "You are a helpful assistant."},]while True:user_input = input("User: ")messages.append({"role": "user", "content": user_input})response = chat_completion_request(messages, tools)if not isinstance(response, requests.Response):print("Error: Failed to generate ChatCompletion response.")breakprint("Model JSON Response: ", response.json()) # 打印模型的 JSON 响应model_response = response.json()["choices"][0]["message"]["content"]print("Model: ", model_response)#messages.append({"role": "assistant", "content": model_response}) # 添加模型的响应到消息历史if "tool_calls" in response.json()["choices"][0]["message"]:function_call = response.json()["choices"][0]["message"]["tool_calls"][0]["function"]if function_call["name"] == "led_light_action":arguments = json.loads(function_call["arguments"])led_light_action(arguments["action"], arguments.get("pin_number"), arguments.get("num_lights"), arguments.get("color"))
6.4添加语音交互
在代码基本功能实现好之后，接下来，来添加交互和页面设计。使用微软的语音服务实现用户语音转文本，以及模型回复文本转语音。
import azure.cognitiveservices.speech as speechsdkfrom azure.cognitiveservices.speech import SpeechConfig, SpeechRecognizer, SpeechSynthesizer, AudioConfig# 初始化语音识别和语音合成服务speech_config = SpeechConfig(subscription=" ", region="eastus") #填入微软语音服务APIrecognizer = SpeechRecognizer(speech_config=speech_config)synthesizer = SpeechSynthesizer(speech_config=speech_config)import speechsdk# 语音转文本函数def recognize_from_microphone():# 配置音频参数audio_config = speechsdk.AudioConfig(use_default_microphone=True)# 创建语音识别器对象speech_recognizer = speechsdk.SpeechRecognizer(speech_config=speech_config, audio_config=audio_config)# 异步进行一次语音识别result = speech_recognizer.recognize_once_async().get()# 根据识别结果进行处理if result.reason == speechsdk.ResultReason.RecognizedSpeech:print("识别的文本为: ", result.text) # 将识别结果打印在终端return result.textelif result.reason == speechsdk.ResultReason.NoMatch:print("无法识别任何语音: {}".format(result.no_match_details))elif result.reason == speechsdk.ResultReason.Canceled:# 处理取消的情况cancellation_details = result.cancellation_detailsprint("语音识别被取消: {}".format(cancellation_details.reason))if cancellation_details.reason == speechsdk.CancellationReason.Error:print("错误详情: {}".format(cancellation_details.error_details))print("你是否设置了语音资源密钥和区域值？")# 文本到语音合成函数def tts(text):# 检查输入文本是否为空if text is None:print("没有文本可合成.")return# 配置语音合成参数speech_config.set_property(property_id=speechsdk.PropertyId.SpeechServiceResponse_RequestSentenceBoundary, value='true')# 配置音频输出参数audio_config = speechsdk.audio.AudioOutputConfig(use_default_speaker=True)# 创建语音合成器对象speech_synthesizer = speechsdk.SpeechSynthesizer(speech_config=speech_config, audio_config=audio_config)# 异步进行文本到语音合成speech_synthesis_result = speech_synthesizer.speak_text_async(text).get()
接下来进行界面设计，添加录音的图片和提示语。
from unihiker import Audiofrom unihiker import GUI# 初始化 Audio 类audio = Audio()# 初始化 GUIu_gui = GUI()u_gui.draw_text(x=120, y=10, text="Start Home Assistant", origin='top', color="blue", font_size=15)u_gui.draw_image(image="mic.jpg", x=120, y=230, w=180, h=50, origin='center', onclick=on_mic_click)u_gui.draw_image(image="Mind.jpg", x=120, y=120, w=350, h=150, origin='center')recording_status = u_gui.draw_text(x=120, y=300, text="Press to start recording", origin='center', color="blue", font_size=15)
可以使用flag变量标记录音状态。
如果 flag 的值为 1，表示用户点击了麦克风按钮，程序将切换到录音状态，并调用 recognize_from_microphone 函数获取用户的语音输入。
如果 flag 的值为 2，表示用户的语音输入已经被处理，程序将调用模型生成回复，并执行相应的工具调用（如果存在）。
如果 flag 的值为 0，表示程序已经处理完用户的输入，准备好接受下一次录音。
flag=0def on_mic_click():global flagflag = 1def main():global flagglobal user_input # 声明 user_input 是一个全局变量user_input = ''u_gui.draw_text(x=120, y=10, text="Start Home Assistant", origin='top', color="blue", font_size=15)u_gui.draw_image(image="mic.jpg", x=120, y=230, w=180, h=50, origin='center', onclick=on_mic_click)u_gui.draw_image(image="Mind.jpg", x=120, y=120, w=350, h=150, origin='center')recording_status = u_gui.draw_text(x=120, y=300, text="Press to start recording", origin='center', color="blue", font_size=15)while True:# 检查flag的值，并更新UIif flag == 1:recording_status.config(text="listening...")user_input = recognize_from_microphone() # 在这里获取用户的输入time.sleep(3)flag = 2 ## 如果用户已经说完了，设置flag的值为2，并发送用户的输入到模型pleaseif user_input != "" and flag == 2:recording_status.config(text="thinking...")conversation.append({"role": "user", "content": user_input})response = chat_completion_request(conversation, tools)print("Model Response: ", response.json())# 将模型的回复转换为语音并播放try:assistant_response = response.json()["choices"][0]["message"]["content"]except KeyError:assistant_response = "我已经成功为您操作"if assistant_response is None:assistant_response="我已经成功为您操作"print(f"Assistant: {assistant_response}")#tts(assistant_response)# 执行模型的回复中的工具调用if 'tool_calls' in response.json()["choices"][0]["message"]:function_call = response.json()["choices"][0]["message"]["tool_calls"][0]["function"]arguments = json.loads(function_call["arguments"])if function_call["name"] == "play_music":play_music(arguments["music"])elif function_call["name"] == "fan_action":fan_action(arguments["action"], arguments.get("pin_number"), arguments.get("user_input"))elif function_call["name"] == "led_light_action":led_light_action(arguments["action"], arguments.get("pin_number"), arguments.get("num_lights"), arguments.get("color"))conversation.append({"role": "assistant", "content": assistant_response})flag = 0 # 设置flag的值为1，表示已经处理完用户的输入recording_status.config(text="Press to start recording")if __name__ == "__main__":main()
七、完整程序
import requests<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="1"> <font class="immersive-translate-loading-spinner notranslate"></font></font>import jsonimport openai<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="2"> <font class="immersive-translate-loading-spinner notranslate"></font></font>import timeimport azure.cognitiveservices.speech as speechsdk<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="3"> <font class="immersive-translate-loading-spinner notranslate"></font></font>from azure.cognitiveservices.speech import SpeechConfig, SpeechRecognizer, SpeechSynthesizer, AudioConfigimport osfrom pinpong.extension.unihiker import * <font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="4"> <font class="immersive-translate-loading-spinner notranslate"></font></font>from unihiker import Audio<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="5"> <font class="immersive-translate-loading-spinner notranslate"></font></font>from unihiker import GUI<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="6"> <font class="immersive-translate-loading-spinner notranslate"></font></font>from pinpong.board import Board, Pin,NeoPixelBoard(" ").begin()# 初始化 Audio 类audio = Audio()u_gui = GUI()flag=0speech_config = SpeechConfig(subscription=" ", region="eastus") <font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="7"> <font class="immersive-translate-loading-spinner notranslate"></font></font>recognizer = SpeechRecognizer(speech_config=speech_config)synthesizer = SpeechSynthesizer(speech_config=speech_config)OPENAI_API_KEY = ' ' <font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="8"> <font class="immersive-translate-loading-spinner notranslate"></font></font>global fan, led, num_beads_global<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="9"> <font class="immersive-translate-loading-spinner notranslate"></font></font>fan, led, num_beads_global = None, None, Nonepin_map = {21: Pin.P21,22: Pin.P22,23: Pin.P23,24: Pin.P24,}COLOR_DICT = {"red": [255, 0, 0],"green": [0, 255, 0],"blue": [0, 0, 255],"white": [255, 255, 255],"black": [0, 0, 0],"yellow": [255, 255, 0],"pink": [255, 105, 180],"purple": [128, 0, 128]}def recognize_from_microphone(): <font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="10"> <font class="immersive-translate-loading-spinner notranslate"></font></font>audio_config = AudioConfig(use_default_microphone=True)speech_recognizer = SpeechRecognizer(speech_config=speech_config, audio_config=audio_config)result = speech_recognizer.recognize_once_async().get()if result.reason == speechsdk.ResultReason.RecognizedSpeech:<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="11"> <font class="immersive-translate-loading-spinner notranslate"></font></font>print("识别的文本为: ", result.text)return result.text<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="12"> <font class="immersive-translate-loading-spinner notranslate"></font></font>elif result.reason == speechsdk.ResultReason.NoMatch:<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="13"> <font class="immersive-translate-loading-spinner notranslate"></font></font>print("No speech could be recognized: {}".format(result.no_match_details)) <font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="14"> <font class="immersive-translate-loading-spinner notranslate"></font></font>elif result.reason == speechsdk.ResultReason.Canceled: <font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="15"> <font class="immersive-translate-loading-spinner notranslate"></font></font>cancellation_details = result.cancellation_details<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="16"> <font class="immersive-translate-loading-spinner notranslate"></font></font>print("Speech Recognition canceled: {}".format(cancellation_details.reason)) <font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="17"> <font class="immersive-translate-loading-spinner notranslate"></font></font>if cancellation_details.reason == speechsdk.CancellationReason.Error:<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="18"> <font class="immersive-translate-loading-spinner notranslate"></font></font>print("Error details: {}".format(cancellation_details.error_details))<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="19"> <font class="immersive-translate-loading-spinner notranslate"></font></font>print("Did you set the speech resource key and region values?") <font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="20"> <font class="immersive-translate-loading-spinner notranslate"></font></font>def tts(text):if text is None:<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="21"> <font class="immersive-translate-loading-spinner notranslate"></font></font>print("No text to synthesize.") <font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="22"> <font class="immersive-translate-loading-spinner notranslate"></font></font>returnspeech_config.set_property(property_id=speechsdk.PropertyId.SpeechServiceResponse_RequestSentenceBoundary, value='true')<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="23"> <font class="immersive-translate-loading-spinner notranslate"></font></font>audio_config = speechsdk.audio.AudioOutputConfig(use_default_speaker=True)speech_synthesizer = speechsdk.SpeechSynthesizer(speech_config=speech_config, audio_config=audio_config)speech_synthesis_result = speech_synthesizer.speak_text_async(text).get()<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="24"> <font class="immersive-translate-loading-spinner notranslate"></font></font>def chat_completion_request(messages, tools=None, tool_choice=None):<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="25"> <font class="immersive-translate-loading-spinner notranslate"></font></font>headers = {"Content-Type": "application/json",<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="26"> <font class="immersive-translate-loading-spinner notranslate"></font></font>"Authorization": "Bearer " + OPENAI_API_KEY, <font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="27"> <font class="immersive-translate-loading-spinner notranslate"></font></font>}json_data = {"model": "gpt-3.5-turbo-1106", "messages": messages} <font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="28"> <font class="immersive-translate-loading-spinner notranslate"></font></font>if tools is not None:<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="29"> <font class="immersive-translate-loading-spinner notranslate"></font></font>json_data.update({"tools": tools}) <font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="30"> <font class="immersive-translate-loading-spinner notranslate"></font></font>if tool_choice is not None:<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="31"> <font class="immersive-translate-loading-spinner notranslate"></font></font>json_data.update({"tool_choice": tool_choice})<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="32"> <font class="immersive-translate-loading-spinner notranslate"></font></font>try:response = requests.post(<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="33"> <font class="immersive-translate-loading-spinner notranslate"></font></font>"https://api.openai.com/v1/chat/completions",headers=headers,json=json_data,)return response<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="34"> <font class="immersive-translate-loading-spinner notranslate"></font></font>except Exception as e:<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="35"> <font class="immersive-translate-loading-spinner notranslate"></font></font>print("Unable to generate ChatCompletion response")<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="36"> <font class="immersive-translate-loading-spinner notranslate"></font></font>print(f"Exception: {e}")<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="37"> <font class="immersive-translate-loading-spinner notranslate"></font></font>return edef play_music(music):if music in ["DADADADUM", "ENTERTAINER", "PRELUDE", "ODE", "NYAN", "RINGTONE", "FUNK", "BLUES", "BIRTHDAY", "WEDDING", "FUNERAL", "PUNCHLINE", "BADDY", "CHASE", "BA_DING", "WAWAWAWAA", "JUMP_UP", "JUMP_DOWN", "POWER_UP", "POWER_DOWN"]:<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="38"> <font class="immersive-translate-loading-spinner notranslate"></font></font>buzzer.play(getattr(buzzer, music), buzzer.OnceInBackground)<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="39"> <font class="immersive-translate-loading-spinner notranslate"></font></font>print(f"已调用，播放{music}")else:print("对不起，我这里没有这个音乐。")def fan_action(action, pin_number=None, user_input=None): <font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="40"> <font class="immersive-translate-loading-spinner notranslate"></font></font>global fan<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="41"> <font class="immersive-translate-loading-spinner notranslate"></font></font>if action == "initialize": <font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="42"> <font class="immersive-translate-loading-spinner notranslate"></font></font>if pin_number not in pin_map:<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="43"> <font class="immersive-translate-loading-spinner notranslate"></font></font>print(f"错误的引脚号：{pin_number}")returnfan = Pin(pin_map[pin_number], Pin.PWM)<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="44"> <font class="immersive-translate-loading-spinner notranslate"></font></font>print(f"风扇已初始化，接在{pin_number}号端口")assistant_response=(f"风扇已初始化，接在{pin_number}号端口")else:if fan is None:<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="45"> <font class="immersive-translate-loading-spinner notranslate"></font></font>print("好的，风扇尚未初始化")returnif "turn on" in user_input:<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="46"> <font class="immersive-translate-loading-spinner notranslate"></font></font>fan.write_analog(800)print("好的，风扇已开启")assistant_response="好的，风扇已开启"elif "turn off" in user_input:<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="47"> <font class="immersive-translate-loading-spinner notranslate"></font></font>fan.write_analog(0)print("好的，风扇已关闭")assistant_response="好的，风扇已关闭"elif "increase" in user_input:<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="48"> <font class="immersive-translate-loading-spinner notranslate"></font></font>fan.write_analog(1023)print("好的，风扇转速已增加")assistant_response="好的，风扇转速已增加"elif "decrease" in user_input: <font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="49"> <font class="immersive-translate-loading-spinner notranslate"></font></font>fan.write_analog(512)print("好的，风扇转速已减少")assistant_response="好的，风扇转速已增加"def led_light_action(action, pin_number=None, num_lights=None, color=None):<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="50"> <font class="immersive-translate-loading-spinner notranslate"></font></font>global ledglobal num_beads_global<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="51"> <font class="immersive-translate-loading-spinner notranslate"></font></font>if action == "initialize":<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="52"> <font class="immersive-translate-loading-spinner notranslate"></font></font>if pin_number is None or num_lights is None:<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="53"> <font class="immersive-translate-loading-spinner notranslate"></font></font>print("错误，灯带'initialize' 动作需要 'pin_number' 和 'num_lights'。")returnled = NeoPixel(Pin(pin_number), num_lights)<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="54"> <font class="immersive-translate-loading-spinner notranslate"></font></font>num_beads_global = num_lights<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="55"> <font class="immersive-translate-loading-spinner notranslate"></font></font>print(f"好的，现在我知道LED 灯已在引脚 {pin_number} 上初始化，有 {num_lights} 个灯珠。")assistant_response=(f"好的，现在我知道LED 灯已在引脚 {pin_number} 上初始化，有 {num_lights} 个灯珠。")elif action == "lightup":if led is None:print("错误：LED 灯尚未初始化。")returnif num_lights is None or color is None:<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="56"> <font class="immersive-translate-loading-spinner notranslate"></font></font>print("错误：'lightup' 动作需要 'num_lights' 和 'color'。告诉我完整的")returnif not isinstance(num_lights, int) or num_lights <= 0 or num_lights > num_beads_global:<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="57"> <font class="immersive-translate-loading-spinner notranslate"></font></font>print("错误：'num_lights' 必须是正整数，且不能大于初始化的灯珠数量。")returncolor_rgb = COLOR_DICT.get(color.lower())<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="58"> <font class="immersive-translate-loading-spinner notranslate"></font></font>if color_rgb is None:<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="59"> <font class="immersive-translate-loading-spinner notranslate"></font></font>print(f"错误：未知颜色 '{color}',请在颜色字典中添加这个颜色")returnfor i in range(num_lights):<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="60"> <font class="immersive-translate-loading-spinner notranslate"></font></font>led[i] = tuple(color_rgb)<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="61"> <font class="immersive-translate-loading-spinner notranslate"></font></font>led.write(i, color_rgb[0], color_rgb[1], color_rgb[2])<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="62"> <font class="immersive-translate-loading-spinner notranslate"></font></font>print(f"好的，我现在已点亮 {num_lights} 个灯珠，颜色为 {color}。")assistant_response=(f"好的，我现在已点亮 {num_lights} 个灯珠，颜色为 {color}。")else:print(f"错误：未知操作 '{action}'")tools = [{"type": "function","function": {"name": "play_music", <font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="63"> <font class="immersive-translate-loading-spinner notranslate"></font></font>"description": "When the user requests to play music, this function is called.Plays music in the house when the user is bored or needs relaxation. The music list includes 'DADADADUM', 'ENTERTAINER', 'PRELUDE', 'ODE', 'NYAN', 'RINGTONE', 'FUNK', 'BLUES', 'BIRTHDAY', 'WEDDING', 'FUNERAL', 'PUNCHLINE', 'BADDY', 'CHASE', 'BA_DING', 'WAWAWAWAA', 'JUMP_UP', 'JUMP_DOWN', 'POWER_UP', 'POWER_DOWN'.",<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="64"> <font class="immersive-translate-loading-spinner notranslate"></font></font>"parameters": {"type": "object","properties": {"music": {"type": "string","enum": ["DADADADUM", "ENTERTAINER", "PRELUDE", "ODE", "NYAN", "RINGTONE", "FUNK", "BLUES", "BIRTHDAY", "WEDDING", "FUNERAL", "PUNCHLINE", "BADDY", "CHASE", "BA_DING", "WAWAWAWAA", "JUMP_UP", "JUMP_DOWN", "POWER_UP", "POWER_DOWN"],"description": "The music to play in the house." <font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="65"> <font class="immersive-translate-loading-spinner notranslate"></font></font>},},"required": ["music"],<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="66"> <font class="immersive-translate-loading-spinner notranslate"></font></font>},}},{"type": "function","function": {"name": "fan_action",<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="67"> <font class="immersive-translate-loading-spinner notranslate"></font></font>"description": "This function initializes or controls the fan based on the given action and user's command. It can initialize the fan, turn on the fan, turn off the fan, increase the fan speed, or decrease the fan speed. It will print an error message if the fan has not been initialized.", <font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="68"> <font class="immersive-translate-loading-spinner notranslate"></font></font>"parameters": {"type": "object","properties": {"action": {"type": "string","enum": ["initialize", "turn_on", "turn_off", "increase_speed", "decrease_speed"],<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="69"> <font class="immersive-translate-loading-spinner notranslate"></font></font>"description": "The action to perform on the fan.",<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="70"> <font class="immersive-translate-loading-spinner notranslate"></font></font>},"pin_number": {"type": "integer","description": "The pin number where the fan is connected. Required for 'initialize' action.", <font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="71"> <font class="immersive-translate-loading-spinner notranslate"></font></font>},"user_input": {"type": "string","description": "The user's command to control the fan. Required for 'turn on', 'turn off', 'increase speed', 'decrease speed' actions.",<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="72"> <font class="immersive-translate-loading-spinner notranslate"></font></font>},},"required": ["action"],<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="73"> <font class="immersive-translate-loading-spinner notranslate"></font></font>},},},{"type": "function","function": {"name": "led_light_action",<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="74"> <font class="immersive-translate-loading-spinner notranslate"></font></font>"description": "This function performs actions on a LED light. It can initialize the LED light on a specific pin with a certain number of beads, or light up a certain number of beads in a specific color.",<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="75"> <font class="immersive-translate-loading-spinner notranslate"></font></font>"parameters": {"type": "object","properties": {"action": {"type": "string","enum": ["initialize", "lightup"],"description": "The action to perform. 'initialize' will set up the LED light on a specific pin with a certain number of beads. 'lightup' will light up a certain number of beads in a specific color."<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="76"> <font class="immersive-translate-loading-spinner notranslate"></font></font>},"pin_number": {"type": "integer","description": "The pin number where the LED light is connected. This is required when the 'action' is 'initialize'."<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="77"> <font class="immersive-translate-loading-spinner notranslate"></font></font>},"num_lights": {<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="78"> <font class="immersive-translate-loading-spinner notranslate"></font></font>"type": "integer","description": "The number of beads to light up or initialize. This is required when the 'action' is 'initialize' or 'lightup'."<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="79"> <font class="immersive-translate-loading-spinner notranslate"></font></font>},"color": {"type": "string","description": "The color to use when lighting up the beads. This is required when the 'action' is 'lightup'."<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="80"> <font class="immersive-translate-loading-spinner notranslate"></font></font>},},"required": ["action"],<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="81"> <font class="immersive-translate-loading-spinner notranslate"></font></font>},},}]conversation = [{"role": "system", "content": "You are a smart home assistant that can play music for relaxation when the user is bored or needs it. You can suggest playing music when the user seems bored or tired.You are a helpful assistant that can also control a fan and a LED light."},<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="82"> <font class="immersive-translate-loading-spinner notranslate"></font></font>]flag=0def on_mic_click():global flagflag = 1def main():global flagglobal user_input # 声明 user_input 是一个全局变量user_input = ''u_gui.draw_text(x=120, y=10, text="Start Home Assistant", origin='top', color="blue", font_size=15)<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="83"> <font class="immersive-translate-loading-spinner notranslate"></font></font>u_gui.draw_image(image="mic.jpg", x=120, y=230, w=180, h=50, origin='center', onclick=on_mic_click)u_gui.draw_image(image="Mind.jpg", x=120, y=120, w=350, h=150, origin='center')recording_status = u_gui.draw_text(x=120, y=300, text="Press to start recording", origin='center', color="blue", font_size=15)<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="84"> <font class="immersive-translate-loading-spinner notranslate"></font></font>while True:if flag == 1:recording_status.config(text="listening...")<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="85"> <font class="immersive-translate-loading-spinner notranslate"></font></font>user_input = recognize_from_microphone()time.sleep(3)flag = 2if user_input != "" and flag == 2:<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="86"> <font class="immersive-translate-loading-spinner notranslate"></font></font>recording_status.config(text="thinking...")<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="87"> <font class="immersive-translate-loading-spinner notranslate"></font></font>conversation.append({"role": "user", "content": user_input})<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="88"> <font class="immersive-translate-loading-spinner notranslate"></font></font>response = chat_completion_request(conversation, tools)<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="89"> <font class="immersive-translate-loading-spinner notranslate"></font></font>print("Model Response: ", response.json())try:assistant_response = response.json()["choices"][0]["message"]["content"]<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="90"> <font class="immersive-translate-loading-spinner notranslate"></font></font>except KeyError:assistant_response = "我已经成功为您操作"if assistant_response is None: <font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="91"> <font class="immersive-translate-loading-spinner notranslate"></font></font>assistant_response="我已经成功为您操作"print(f"Assistant: {assistant_response}")<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="92"> <font class="immersive-translate-loading-spinner notranslate"></font></font>if 'tool_calls' in response.json()["choices"][0]["message"]: <font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="93"> <font class="immersive-translate-loading-spinner notranslate"></font></font>function_call = response.json()["choices"][0]["message"]["tool_calls"][0]["function"]<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="94"> <font class="immersive-translate-loading-spinner notranslate"></font></font>arguments = json.loads(function_call["arguments"])<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="95"> <font class="immersive-translate-loading-spinner notranslate"></font></font>if function_call["name"] == "play_music":<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="96"> <font class="immersive-translate-loading-spinner notranslate"></font></font>play_music(arguments["music"])<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="97"> <font class="immersive-translate-loading-spinner notranslate"></font></font>elif function_call["name"] == "fan_action": <font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="98"> <font class="immersive-translate-loading-spinner notranslate"></font></font>fan_action(arguments["action"], arguments.get("pin_number"), arguments.get("user_input"))<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="99"> <font class="immersive-translate-loading-spinner notranslate"></font></font>elif function_call["name"] == "led_light_action":<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="100"> <font class="immersive-translate-loading-spinner notranslate"></font></font>led_light_action(arguments["action"], arguments.get("pin_number"), arguments.get("num_lights"), arguments.get("color"))<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="101"> <font class="immersive-translate-loading-spinner notranslate"></font></font>conversation.append({"role": "assistant", "content": assistant_response})<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="102"> <font class="immersive-translate-loading-spinner notranslate"></font></font>flag = 0recording_status.config(text="Press to start recording") <font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="103"> <font class="immersive-translate-loading-spinner notranslate"></font></font>if __name__ == "__main__":<font class="notranslate immersive-translate-target-wrapper" translate="no" lang="zh-CN" data-immersive-translate-loading-id="104"> <font class="immersive-translate-loading-spinner notranslate"></font></font>main()
参考资料:
OpenAI官方文档
https://platform.openai.com/docs/guides/function-calling
行空板的板载硬件库:
https://www.unihiker.com.cn/wiki/pinpong_python_lib
ChatGLM3-6B funtion calling官方文档:
https://github.com/THUDM/ChatGLM3/blob/main/tools_using_demo/README.md
智谱AI技术文档:
https://zhipu-ai.feishu.cn/wiki/YInmwPmyii67VRkzU3BchPNzncg
往期推荐