How to implement the grayscale matching algorithm in pyautogui?

2 years ago
William Carter
2 minutes

To implement the pyautogui grayscale matching algorithm, you can follow these steps:

  1. Firstly, convert both the image to be matched and the reference image to grayscale. You can use the cv2.cvtColor() function from the OpenCV library to convert the image to grayscale.
  2. Next, use the screenshot() function in the pyautogui library to capture a screen shot. Also, convert the screenshot to a grayscale image.
  3. Utilize the template matching algorithm on grayscale images. You can use the cv2.matchTemplate() function from the OpenCV library to perform template matching. This function slides a template image of the same size as the reference image across a screenshot and calculates the level of match at each position.
  4. Find the best matching position based on the degree of similarity. You can use the cv2.minMaxLoc() function from the OpenCV library to get the minimum and maximum values of the match and find the position corresponding to the maximum value.
  5. Lastly, the locateOnScreen() function from the pyautogui library can be used to locate images on the screen. This function searches for areas similar to the reference image in a screen capture and returns their location.

Here is a simple example code:

import cv2
import pyautogui

# 将图像转换为灰度图像
def convert_to_gray(image):
    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    return gray

# 获取屏幕截图并转换为灰度图像
screenshot = pyautogui.screenshot()
screenshot_gray = convert_to_gray(screenshot)

# 加载参考图像并转换为灰度图像
reference_image = cv2.imread('reference_image.jpg')
reference_image_gray = convert_to_gray(reference_image)

# 进行模板匹配
result = cv2.matchTemplate(screenshot_gray, reference_image_gray, cv2.TM_CCOEFF_NORMED)

# 获取匹配程度的最大值和位置
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(result)
top_left = max_loc
bottom_right = (top_left[0] + reference_image_gray.shape[1], top_left[1] + reference_image_gray.shape[0])

# 在屏幕截图上绘制矩形框来标识匹配位置
cv2.rectangle(screenshot, top_left, bottom_right, (0, 255, 0), 2)

# 显示结果
cv2.imshow('Result', screenshot)
cv2.waitKey(0)
cv2.destroyAllWindows()

In the code above, we first convert both the screen capture and the reference image to grayscale images, then use the cv2.matchTemplate() function for template matching to find the best match position, and draw a rectangle on the screen capture to identify the match position. Finally, we use the cv2.imshow() function to display the result.

bannerAds