This topic demonstrates how to use CV mode to interact with a software interface, using sending a message in the DingTalk client as an example.
Overview
Some software interfaces, such as those in instant messaging (IM) clients, do not expose their UI structure for automation. In these cases, you must use CV mode to interact with the software interface. This example demonstrates how to send a message to a specific contact in the DingTalk client.
Implementation approach
This example uses DingTalk client version 8.3.10. The implementation is for reference only, as the client's appearance may vary due to theme settings or software updates.
This example only covers the minimum end-to-end process for sending a message. In production, similar scenarios often require handling more complex details and edge cases.
The procedure and the captured CV controls are as follows:
Enter the full name of the target contact in the search box at the top of the DingTalk client. The CV control for this action is shown below, with the anchor positioned in the center.
After entering the name, press the Enter key to open the contact's chat window.
To enter the message, use a CV control anchored to the send button to find the text input area. As shown in the image below, the anchor is positioned outside the target image.
After entering your message, click the send button. The CV control for this action is shown below, with the anchor in the center.
Code-based development
The following code implements this procedure:
from rpa.core import *
from rpa.utils import *
import rpa4 as rpa # Use the V4 engine
import time
def start():
# Activate the DingTalk client window
wnd = rpa.ui.win32.catch('DingTalk',mode='exact',process_name='DingTalk')
wnd.activate()
# Enter the contact's full name in the search box
rpa.ai.cv.element.input_text("search","Helen")
# Wait for the client to load and press Enter to open the chat window
sleep(1)
rpa.ui.win32.send_key("{ENTER}")
sleep(1)
# Enter the message in the chat box
rpa.ai.cv.element.input_text("input","hello")
# Click the send button
rpa.ai.cv.element.click("send")Visual development
The workflow is as follows:
