一、模型编辑
将该文件解压缩,全部复制到柚子相机的APP文件夹内。一定要删除原先的APP文件夹内的所有文件!!!删除前现将原先APP文件夹内文件备份!!!不过备份时会出现无法复制的情况,结束电脑的一些进程,或者重启电脑多试几次。还是无法复制就全部删了,联系厂家要原始文件。
![image.png](https://makelogimg.dfrobot.com.cn/makelog/60abce20aa613609f270a04c/aa62bb04971f6bd3dc78234623d7d7e5.png)
用Visual Studio Code软件打开main文件,对文件做基本修改柚子相机才能使用手势检测功能。修改的第一部分为串口波特率和串口数据初始化,修改的第二部分为串口发送数据。第二部分修改后mind+就可以调用相机的数据了。
![image.png](https://makelogimg.dfrobot.com.cn/makelog/60abce20aa613609f270a04c/2443ec7f58c6a21c23b96653e622b926.png)
![image.png](https://makelogimg.dfrobot.com.cn/makelog/60abce20aa613609f270a04c/10e22bf5d454d354588fb3899be75c5c.png)
串口波特率和串口数据初始化
![image.png](https://makelogimg.dfrobot.com.cn/makelog/60abce20aa613609f270a04c/452661315387e297689ee53746f97ea1.png)
串口发送数据
全部的main.py代码
代码
from time import time
from maix import serial #导入库
ser = serial.Serial("/dev/ttyS1",115200) #设置端口
ser_str="" #数据初始化
ser_sendstr=""
ser_num = 0
class Hand:
mud_path = "./hand_int8.mud"
labels = ["0","1","2","3","4","5"]
anchors = [3.78, 5.81, 3.97, 3.98, 4.05, 4.98, 4.81, 5.41, 2.91, 4.53]
def __init__(self) -> None:
from maix import nn
self.model = nn.load(self.mud_path)
from maix.nn import decoder
self.decoder = decoder.Yolo2(len(self.labels) , self.anchors , net_in_size = (224, 224) ,net_out_size = (7,7))
def __del__(self):
del self.model
del self.decoder
def cal_fps(self ,start , end):
one_second = 1
one_flash = end - start
fps = one_second / one_flash
return fps
def draw_rectangle_with_title(self ,img, box, disp_str , fps ):
img.draw_rectangle(box[0], box[1], box[0] + box[2], box[1] + box[3],color=(255, 0, 0), thickness=2)
img.draw_string(box[0], box[1]+ box[3] ,disp_str, scale=1,color=(0, 0, 255), thickness=2)
img.draw_string(0, 0 ,'FPS :'+str(fps), scale=2 ,color=(0, 0, 255), thickness=2)
def uart_send(category,content,x="0",y= "0",size ="0",confidence ="0"):
global ser_str,ser_sendstr,ser_num,ser
ser_num +=1
if ser_num >=30:
ser_str = str(category)+"*"+"x:"+str(x)+"*"+"y:"+str(y)+"*"+str(size)+"*"+str(content)+"*"+str(confidence)+"*#"
ser_sendstr = "YZ*"+str(len(ser_str))+"*"+ser_str
ser.write(ser_sendstr.encode("utf-8"))
ser_num = 0
def process(self,input):
t = time()
out = self.model.forward(input, quantize=1, layout = "hwc")
boxes, probs = self.decoder.run(out, nms=0.5, threshold=0.6, img_size=(224,224))
for i, box in enumerate(boxes):
class_id = probs[i][0]
prob = probs[i][1][class_id]
disp_str = "{}:{:.2f}%".format(self.labels[class_id], prob*100)
fps = self.cal_fps(t, time())
self.draw_rectangle_with_title(input, box, disp_str, fps)
self.uart_send("handnum",self.labels[class_id],confidence =prob*100)
def main():
from maix import display, camera
global Hand
app = Hand()
while True:
img = camera.capture().resize(size=(224,224))
app.process(img)
display.show(img)
main()
三、Mind+编程
1.程序代码
![image.png](https://makelogimg.dfrobot.com.cn/makelog/60abce20aa613609f270a04c/e07d56b1371ad3ef12769df6f415b70f.png)
2. 串口数据
![image.png](https://makelogimg.dfrobot.com.cn/makelog/60abce20aa613609f270a04c/cf6522538b95b95a393ee3535cd0c094.png)
可以看到串口数据不是太稳定,实际相机识别时板载LED灯交替闪烁。相机检测时,数据本身不太稳定,从相机屏幕就可以看出。
四、使用相机
相机连接电脑,检测手势。数字0、1、2、3时,手离相机近一点。数字4和5时手离相机远些检测。
![image.png](https://makelogimg.dfrobot.com.cn/makelog/60abce20aa613609f270a04c/426f9a4231cfb5d61a5d0617175806d1.png)
![image.png](https://makelogimg.dfrobot.com.cn/makelog/60abce20aa613609f270a04c/c3c7c165b56746bbe5fcf4699ebd514b.png)
![image.png](https://makelogimg.dfrobot.com.cn/makelog/60abce20aa613609f270a04c/e27e088f5a5355903836417c34fdd916.png)
![image.png](https://makelogimg.dfrobot.com.cn/makelog/60abce20aa613609f270a04c/2b8fd9563ebfef65341f4bfd795755f1.png)
![image.png](https://makelogimg.dfrobot.com.cn/makelog/60abce20aa613609f270a04c/5f864a6bd6121203ca91f51c7da2b1fb.png)
柚子相机配上其他执行器,看官方案例,执行器的接口似乎现有的线接不上,还没试。接执行器难度不大,麻烦的地方是模型的训练、部署。
附件
DeadWalking2024.05.27
学习一下,用掌控相机!