所有分类
主题 主题
平台 平台
我的工作台
userHead
注册时间 [[userInfo.create_time]]
创造力 [[userInfo.creativity]]
[[userInfo.remark]]
[[d.project_title]]
articleThumb
[[d.material_name]]
timelineThumb
进入工作台
折叠
所有分类 我的工作台
展开

【科学探究】串口数据记录器——分析数据判断手势

云天 云天 2022-05-08 20:21:37

利用Mediapipe获取手势数据,通过串口数据记录器存储数据,使用EXCEL分析数据,再利用程序分析手势。

【测试数据传输】

利用显示屏显示接收的数据

①在Mind+的扩展库、功能模块,添加“软串口”。

projectImage

②拖曳积木编写程序

projectImage

代码第四行设置主控板11号引脚为Rx,6号引脚为Tx。

数据传输测试(Python)

代码 代码
	                    					import serial
import time
ser = serial.Serial('COM4', 9600, timeout=0)
Timestamp = str(int(time.time()))
if __name__ == '__main__':
     #while True:
     for index in range(10):
                   Timestamp = str(int(time.time()))
                   time.sleep(2)
                   data=Timestamp+"s"
                   
                   ser.write(data.encode("utf-8"))#向端口些数据
                   #print(data.encode("utf-8"))
     ser.close()#关闭端口
	                    				
projectImage

手部的节点,每只手有21个。

projectImage

【存储数据】

代码 代码
	                    					import cv2
import numpy as np
import mediapipe as mp
import serial
import time
ser = serial.Serial('COM4', 9600, timeout=0)
mp_drawing = mp.solutions.drawing_utils
mp_holistic = mp.solutions.holistic

joint_list = [[6,9, 10],[4,3, 2],[8, 7, 6], [12, 11, 10], [16, 15, 14], [20, 19, 18]]  # 手指关节序列

cap = cv2.VideoCapture(0)
Timestamp = time.time()
with mp_holistic.Holistic(
        min_detection_confidence=0.5,
        min_tracking_confidence=0.5) as holistic:
    while cap.isOpened():
        data=""
        success, image = cap.read()
        if not success:
            print("Ignoring empty camera frame.")
            break
        image.flags.writeable = False
        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
        results = holistic.process(image)
        image.flags.writeable = True
        image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)

        # 渲染
        #mp_drawing.draw_landmarks(image,results.face_landmarks)
        #mp_drawing.draw_landmarks(image,results.pose_landmarks)

        mp_drawing.draw_landmarks(image, results.left_hand_landmarks)
        mp_drawing.draw_landmarks(image, results.right_hand_landmarks)
        # 监测到右手,执行
        if results.right_hand_landmarks:
            RHL = results.right_hand_landmarks
            # 计算角度
            for joint in joint_list:
                a = np.array([RHL.landmark[joint[0]].x, RHL.landmark[joint[0]].y])
                b = np.array([RHL.landmark[joint[1]].x, RHL.landmark[joint[1]].y])
                c = np.array([RHL.landmark[joint[2]].x, RHL.landmark[joint[2]].y])
                # 计算弧度
                radians_fingers = np.arctan2(c[1] - b[1], c[0] - b[0]) - np.arctan2(a[1] - b[1], a[0] - b[0])
                angle = np.abs(radians_fingers * 180.0 / np.pi)  # 弧度转角度

                if angle > 180.0:
                    angle = 360 - angle
                if data!="":
                 data=data + "," + str(angle)
                else:
                 data=str(angle)
                cv2.putText(image, str(round(angle, 2)), tuple(np.multiply(b, [640, 480]).astype(int)),
                            cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2, cv2.LINE_AA)
                
        if results.left_hand_landmarks:
            RHL = results.left_hand_landmarks
            # 计算角度
            for joint in joint_list:
                a = np.array([RHL.landmark[joint[0]].x, RHL.landmark[joint[0]].y])
                b = np.array([RHL.landmark[joint[1]].x, RHL.landmark[joint[1]].y])
                c = np.array([RHL.landmark[joint[2]].x, RHL.landmark[joint[2]].y])
                # 计算弧度
                radians_fingers = np.arctan2(c[1] - b[1], c[0] - b[0]) - np.arctan2(a[1] - b[1], a[0] - b[0])
                angle = np.abs(radians_fingers * 180.0 / np.pi)  # 弧度转角度

                if angle > 180.0:
                    angle = 360 - angle

                cv2.putText(image, str(round(angle, 2)), tuple(np.multiply(b, [640, 480]).astype(int)),
                            cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2, cv2.LINE_AA)
                if joint[0]==6:
                    if angle>50 and time.time()-Timestamp>2:
                        Timestamp=time.time()
                        data=data+"s"
                        ser.write(data.encode("utf-8"))#向端口些数据
                        data=""

        # cv2.imshow('MediaPipe Holistic', cv2.flip(image, 1))
        cv2.imshow('Mediapipe Holistic', image)  # 取消镜面翻转
        if cv2.waitKey(5) == ord('q'):
            break
cap.release()

	                    				

【硬件连接】

 

projectImage

【发送数据】

右手食指与中食打开角度超过50度,开始向主板发送数据。

projectImage

【处理数据】

串口数据记录器用数据线连接电脑,拷贝数据文件。

projectImage
projectImage

将数据文件TXT修改为CSV,用EXCE程序打开。并利用公式计算最大值(MAX)和最小值(MIN)。

projectImage

通过以上方法,可记录各种手势的数据,通过EXCEL进行分析,再利用Python程序对手势进行相应判断。

projectImage

【手势判断】

利用所得数据编写程序

代码 代码
	                    					                if joint[0]==4:
                    if angle>162 and angle<180:
                      palm=palm+1
                if joint[0]==8:
                    if angle>175 and angle<179:
                      palm=palm+1
                if joint[0]==12:
                    if angle>178 and angle<180:
                      palm=palm+1 
                if joint[0]==16:
                    if angle>177 and angle<180:
                      palm=palm+1
                if joint[0]==20:
                    if angle>175 and angle<179:
                      palm=palm+1
                
                if joint[0]==4:
                    if angle>112 and angle<133:
                      fist=fist+1
                if joint[0]==8:
                    if angle>126 and angle<180:
                      fist=fist+1
                if joint[0]==12:
                    if angle>118 and angle<179:
                     fist=fist+1
                if joint[0]==16:
                    if angle>118 and angle<180:
                      fist=fist+1
                if joint[0]==20:
                    if angle>130 and angle<176:
                      fist=fist+1
            if palm==5:
                cv2.putText(image, "PALM", (100,100),cv2.FONT_HERSHEY_SIMPLEX, 2, (255, 255, 255), 2, cv2.LINE_AA)
            if fist==5:
                cv2.putText(image, "FIST", (100,100),cv2.FONT_HERSHEY_SIMPLEX, 2, (255, 255, 255), 2, cv2.LINE_AA)
    
	                    				

【完整程序】

代码 代码
	                    					import cv2
import numpy as np
import mediapipe as mp
import serial
import time
ser = serial.Serial('COM4', 9600, timeout=0)
mp_drawing = mp.solutions.drawing_utils
mp_holistic = mp.solutions.holistic

joint_list = [[6,9, 10],[4,3, 2],[8, 7, 6], [12, 11, 10], [16, 15, 14], [20, 19, 18]]  # 手指关节序列

cap = cv2.VideoCapture(0)
Timestamp = time.time()
palm=0
fist=0
with mp_holistic.Holistic(
        min_detection_confidence=0.5,
        min_tracking_confidence=0.5) as holistic:
    while cap.isOpened():
        
        success, image = cap.read()
        if not success:
            print("Ignoring empty camera frame.")
            break
        image.flags.writeable = False
        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
        results = holistic.process(image)
        image.flags.writeable = True
        image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)

        # 渲染
        #mp_drawing.draw_landmarks(image,results.face_landmarks)
        #mp_drawing.draw_landmarks(image,results.pose_landmarks)

        mp_drawing.draw_landmarks(image, results.left_hand_landmarks)
        mp_drawing.draw_landmarks(image, results.right_hand_landmarks)
        # 监测到右手,执行
        if results.right_hand_landmarks:
            RHL = results.right_hand_landmarks
            palm=0
            fist=0
            # 计算角度
            for joint in joint_list:
                a = np.array([RHL.landmark[joint[0]].x, RHL.landmark[joint[0]].y])
                b = np.array([RHL.landmark[joint[1]].x, RHL.landmark[joint[1]].y])
                c = np.array([RHL.landmark[joint[2]].x, RHL.landmark[joint[2]].y])
                # 计算弧度
                radians_fingers = np.arctan2(c[1] - b[1], c[0] - b[0]) - np.arctan2(a[1] - b[1], a[0] - b[0])
                angle = np.abs(radians_fingers * 180.0 / np.pi)  # 弧度转角度

                if angle > 180.0:
                    angle = 360 - angle
             
                cv2.putText(image, str(round(angle, 2)), tuple(np.multiply(b, [640, 480]).astype(int)),
                            cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2, cv2.LINE_AA)
                if joint[0]==4:
                    if angle>162 and angle<180:
                      palm=palm+1
                if joint[0]==8:
                    if angle>175 and angle<179:
                      palm=palm+1
                if joint[0]==12:
                    if angle>178 and angle<180:
                      palm=palm+1 
                if joint[0]==16:
                    if angle>177 and angle<180:
                      palm=palm+1
                if joint[0]==20:
                    if angle>175 and angle<179:
                      palm=palm+1
                
                if joint[0]==4:
                    if angle>112 and angle<133:
                      fist=fist+1
                if joint[0]==8:
                    if angle>126 and angle<180:
                      fist=fist+1
                if joint[0]==12:
                    if angle>118 and angle<179:
                     fist=fist+1
                if joint[0]==16:
                    if angle>118 and angle<180:
                      fist=fist+1
                if joint[0]==20:
                    if angle>130 and angle<176:
                      fist=fist+1
            if palm==5:
                cv2.putText(image, "PALM", (100,100),cv2.FONT_HERSHEY_SIMPLEX, 2, (255, 255, 255), 2, cv2.LINE_AA)
            if fist==5:
                cv2.putText(image, "FIST", (100,100),cv2.FONT_HERSHEY_SIMPLEX, 2, (255, 255, 255), 2, cv2.LINE_AA)
       
        if results.left_hand_landmarks:
            RHL = results.left_hand_landmarks
            
            # 计算角度

            for joint in joint_list:
                a = np.array([RHL.landmark[joint[0]].x, RHL.landmark[joint[0]].y])
                b = np.array([RHL.landmark[joint[1]].x, RHL.landmark[joint[1]].y])
                c = np.array([RHL.landmark[joint[2]].x, RHL.landmark[joint[2]].y])
                # 计算弧度
                radians_fingers = np.arctan2(c[1] - b[1], c[0] - b[0]) - np.arctan2(a[1] - b[1], a[0] - b[0])
                angle = np.abs(radians_fingers * 180.0 / np.pi)  # 弧度转角度

                if angle > 180.0:
                    angle = 360 - angle

                cv2.putText(image, str(round(angle, 2)), tuple(np.multiply(b, [640, 480]).astype(int)),
                            cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2, cv2.LINE_AA)
                # cv2.imshow('MediaPipe Holistic', cv2.flip(image, 1))
        cv2.imshow('Mediapipe Holistic', image)  # 取消镜面翻转
        if cv2.waitKey(5) == ord('q'):
            break
cap.release()

	                    				
Makelog作者原创文章,未经授权禁止转载。
0
0
评论
[[c.user_name]] [[c.create_time]]
[[c.parent_comment.count]]
|
[[c.comment_content]]