What is Surroga?
Surroga is a keyboard & mouse automation utility for macOS. It can automate every thing you can do with your Mac, even iPhone.
Surroga can be downloaded on AppStore. We continue to make Surroga the best automation utility ever to fit you needs. We need you help to shape the future direction of new features and improvements. If you have any comments about Surroga, please feel free to let us know. , we will reply every feedback email.
- What is Surroga?
- Script
- Statement
- Control Flow
- Condition
- Actions
- Variables
- Examples
- Log
- Performance
- Expert Mode
- Multi-Display
- Background Task
- In-App Purchases
- Privacy & Security
- Privacy Statement
What is Surroga?
Surroga is a keyboard & mouse automation utility for macOS.- Send keystrokes or typing.
- Move, click, and drag anything via mouse.
- Calculate numbers.
- Text recognize & process.
- Image recognize & click.
- Flexible screenshots.
- Flexible copy & paste.
- Conditional delay.
- Open 3rd-part Apps.
- Call terminal commands.
- Nested scripts and loops.
- Variables
- Conditions
- Programmable UI, include sound, flash-screen and pop-up.
Script
Surrga can record user operations in the specified window.
- Open the left sidebar, click
(Record), then choice the application to record. - Do mouse operations in the window of the selected application.
- Click
(Stop), or press Shift-Command-R, or press Shift-Command-ESC, to stop recording. - Enter the name of the new script. then click Save.
The new recorded script will appear in the script list at the left sidebar.
To remove a script, select it via the script list at left sidebar, then press the DELETE key on the keyboard. Or, click the script via the mouse right button, the click the Delete option in the context menu.
Select a script, then click 
, choice Share or Share as Protected. The protected script is not editable, and will not create the logs.
Import a script.
Statement
Statement group expressions and control the flow of execution.
Insert, Sort, Remove a Statement
Insert a Statement
Click on + on the toolbar in the navigation bar of the script editor, select and add a new statement at the end of the script. If you want to insert the new statement at the specified statement, you can right-click on that statement to invoke the context menu, and then select New.
Sort Statements
Reorder statements with simple drag & drop.
Delete a Statement
In the script editor, click a statement via the mouse right button, the click the Delete option in the context menu.
Error Handling
When a statement encounters an error, we can specify how to handle it.
- Ignore - Execute the next statement.
- Return - Abandon the remaining statements and return to the previous script (parent script)
- Abort - Stop Running
Control Flow
'If', 'Then', and 'Else' are collectively referred to as 'conditional statements'.
By default, one statement include only one action. When a statement is executed, this action will be executed.
On the other side, we can insert an IF sub-statement into a statement. The IF sub-statement is consist of one or more Conditions . The action of the statement, so called THEN sub-statement, will be executed only when conditions are true. If Conditions are false, this statement can be skiped, or execute the action in the ELSE sub-statement.
AND, OR
There are two combination types that be used between multiple conditions in the same statement.
- AND: The actions contained in a statement will only be executed when all conditions of that statement are met.
- OR: When any one of the conditions in a statement is met, the actions contained in that statement will be executed.
When a statement contains multiple conditions, we can setup their combination type:
Surroga doesn't support LOOP statement yet. Instead, it use multiple methods to implement flexible loop.
Script Repeating
Repeating at script level by Repeat-limited, Repeat-infinite, Repeat-until or While-Do.
- Never: Never repeat the current script.
- Infinite: Repeat the current script infinitly.
- Limited: Repeat the current script the specified number of times.
- Repeat-Until：Repeats the current script until its condition is true. The test is done after the last statement of the script, so the script is always executed at least once.
- While-Do：If the condition is false, then the loop ends; otherwise, it executes the current script and repeats the process. The test is done before the first statement of the script, so the script may be skipped at beginning.
To initialize a loop, consider two scripts, main and sub-script. In the main script, we initialize the variable name VAR by assign a value(1) to it before call the sub-script. In the sub-script, it increase the VAR by 1 each time, and repeat until VAR is greater than 5. Surroga ship with a on-board script named "Counting". To view the results of this script, open a window that support text inpurt (typing keys), the click the system status bar -> Surroga -> Run -> General -> Counting.
Continue Statement
Bypass the remainng statements in the current script, move on to the beginning of the current script, re-execute the current script from the beginning.
In Surroga, the Continue statement is implemented as a statement containing one or more conditions and a Continue action, and is usually used to perform script-level loops under specific conditions.
Call another script. At this time, the caller is often referred to as the parent script. The called script is referred to as the child script.
RETURN statement will abort the current script, and return to its parent script (the caller).
RETURN statement doesn't returns any value to its parent script. Instead, if we want to do this, assign the value to one of variables before return, then read the value of the variable in the parent script. In Surroga, the scope of variables are global. Variables are shared by all scripts.
In Surroga, it is implemented as an action.
EXIT, also called ABORT, END or STOP statement. This statement only only abort the running of the current script, but also its parent script, ..., parent of parent. Finally, it abort the 'Root' script.
In Surroga, it is implemented as an action.
Condition
Get the pixel color at a specified position in the window and compare it with a specified color. If they are similar, it is judged to be true. Otherwise, it is judged to be false.
If both items being compared are colors and are similar, then it is judged to be true. Otherwise, it is judged to be false.
If both items being compared are images and are similar, then it is judged as true. Otherwise, it is judged as false.
If the image specified by the former contains the text specified by the latter, then it is judged to be true. Otherwise, it is judged to be false.
Actions
Type a single keystroke, pretty much any kind of keystroke you can type on the keyboard.
How to specify the keyTo specify the desired key to press, open the action editor, expand the action details, and click the key specified by the action with the mouse; that key will be highlighted. Then press a key on the hardware keyboard. The editor will record the pressed key and assign it as the key for that action.
Tips: The macOS keyboard shortcuts can not be recorded. For example, Option–VolumeDecrease on Apple Keyboard, or Alt–F11 on Windows keyboards:
- Can not be recorded in macOS 26.2.
- Can be recorded in macOS 15.7.3.
Therefore, Surroga provides some pre-made key actions to insert them. If you find that Surroga can not support inserting a certain specific key combination, please .
For certain special scenarios or specific applications, special dedicated hardware is required to assist with keyboard operations. At this time, the Hardware Assist option needs to be enabled in the action editor. For example, due to macOS system features, the action of 'simulating pressing the F11 key' can only work properly when hardware assistance is enabled.
Continuously press a series of keys on the physical keyboard in the predetermined order and speed to automatically input a piece of text.
Hardware Assisted
For certain special scenarios or specific applications, special dedicated hardware is required to assist with keyboard operations. At this time, the Hardware Assist option needs to be enabled in the action editor (Tip: You must enable Expert Mode first to see this option).
Move the mouse cursor, click the mouse button, or dragging.
Move the mouse cursor to
- text in window / on screen.
- color in window.
- image in window.
- point at screen or in window.
- relative to the current position.
Click the mouse button at the calculated position.
- Click on text in window / on screen.
- Click on color in window.
- Click on image in window.
- Click at point in window / on screen.
- Click at the current mouse cursor position with offset.
Similarity
In the process of searching for images and colors, Surroga will calculate the similarity between each 'suspected' object and the target object. If the similarity is greater than the specified threshold, the match is successful.
Therefore, in operations like searching for images or colors, the higher the similarity parameter, the more similar the found objects are to the target object, but the less results. If a mismatch result is found, it is recommended to raise the similarity threshold. If nothing can be found, it is suggested to lower the similarity threshold.
Locate the Subimage
To accelerate the image recognizing,
- This sub-image should be generated using the capture function in editor, rather than a previously saved screenshot.
- We should use the smallest size of the sub-image to search for. The suggested size is 32 X 32. The screenshot size can be adjusted through Settings > Advanced > Capture Size.
- The sub-image should contains unique content other than the remains in the original image
- The color of the top-left pixel of the sub-image should be the least color in all pixels of the original image.
Tips: If the subgraph is a foreground object with an irregular contour, its exposure to the background should be minimized.
Incorrect example: As the target prospect being searched for, the tile reveals the background image underneath.
Correct example: As the target to be searched, it only contains the pixels of the foreground object, without revealing the background image.
Special case: As a foreground target to be searched, its filled content lacks uniqueness and can only rely on shape to differentiate the target object from interference objects. If the background color may change, you can use 'Replace Color in Image' for preprocessing to keep the background color consistent.
Extra Match
When performing mouse operations on text and images, in order to handle cases where the window contains multiple target objects (texts, images), some additional matching criteria can be used to filter out irrelevant targets. For example, we may require that the target object is nearby another specified text or image.
As shown in the figure above, we want to click on green 'Coordinates', but there are three instances of 'Coordinates' in the image. To eliminate interference, we can set it to only click on the 'Coordinates' that is near the 'Screen'.
Scattered
Once the target point is specified or found, use that point as the base point. Generate a randomly offset point around the base point. The maximum distance between the offset point and the base point should not exceed the scatter radius. Example: randomly click near the coordinates (500, 500).
This feature is used to simulate the randomness of mouse operations by human.
Surroga supports flexible and diverse wait actions.
- Limited Wait：Wait for a fixed duration.
- Random Wait: Wait for a random duration.
- Conditional Wait: Wait until the specified conditions are true.
Tips: When waiting for a fixed duration, if you find that the actual waiting time far exceeds the preset value, it is usually because the macOS 'App Nap' mechanism has been triggered. To resolve this issue, please refer to macOS App Nap.
Take a screenshot of screen or window.
We can specify the content to capture:
- Entire Screen
- Selected Portion of Screen
- Selected Window.
- Selected Portion of Selected Window
To specify an region (portion of screen or window), you need to define the position and size of that region. The origin is at the top left corner of the screen or window. When specifying a screen region, what is specified is the absolute position of that region on the screen. When specifying a window region, what is specified is the relative position of that region within the window. For a region which size is (100,100):
- the top-left corner of the window: (X: 0, Y: 0, W: 100, H: 100)
- the top-right corner of the window: (-100, 0, 100, 100)
- the bottom-left corner of the window: (0, -100, 100, 100)
- the bottom-right corner of the window: (-100, -100, 100, 100)
Window Title
The most macOS app shows more than one windows at the same time. For example, if we click the about option of an app, this app will shows at least two windows, the main window and the about window. For Safari, every Tab own its window. We can specify the Window Title to distinguish these windows. For Safari, the window title, in most cases, is the title of Tab.
Recognize text from image or content in window.
Obviously, it's possible for an image to contain more than one piece of text. So, the results of recognizing-text is array of text, instead of just one text in a basic variable. For example, for an image that only contains a number, 32, this actions returns the array ["32"]. Then, use Get the First Element in Array action to get 32。
The result of recognization may be different according to recognization language. For example, if we setup the recognization language to English, it can not recognize Chinese in the content.
Text-to-Speech (TTS) can produces synthesized speech from text in various languages, thereby supporting automated voice control, making it particularly suitable for scenarios that require voice control (such as certain iPhone applications).
There is a set of actions specifically used to calculate 2D coordinate points. In these actions, variables are used to store a 2D coordinate point, also known as a coordinate point variable.
- Get x-coor of point: var2 = var1.x
- Get y-coor of point: var2 = var1.y
- Set x-coor of point: var1.x = var2
- Set y-coor of point: var1.y = var2
- Offset a point: var2 = var1.offset(x, y)
- Get distance between two points: var3 = distance(var1, var2)
- Get midpoint between two points: var3 = midpoint(var1, var2)
Pop-up a modal dialog to show something or input something.
Send a local notification that showed on the current device. Notifications are used to show the status of running script, or notify users to do necessary operations.
This action will print a log message. The log entry will appear in the log window on the left sidebar or in the log file (if file logging is enabled).
When displayed in the left sidebar window, log entries of different levels are shown in different colors, making them more noticeable. This parameter does not apply to file logs.
Head Up Display, HUD, overlays text or images on the main screen.
- Absolute positioning. The coordinate system takes the bottom-left corner of the screen as the origin (0,0).
- The content displayed will stay visible on the screen. We can invoke HUD action again to modify the displayed content, style, and position. Invoke the Cancel HUD action or exit the Surroga to cancel the display.
- Currently, it supports displaying up to three text, three images and three rectangles at the same time. Therefore, when showing or hiding, it is necessary to specify which object (view) is being operated on.
Tips: We can use this feature to overlay some auxiliary information on the screen. For example, use HUD text to show the number of battles, the number of victories, and the total number of coins collected, use HUD rectangle to show a process bar.
This action is used to save the content of the variable to a file.
- Support text, images, and text-based numerical values.
- If no filename is specified, the current time will be used as the default filename.
- Supports basic variables only. If you want to save arrays or dictionaries, you can first encode the array or dictionarie into a basic variables by the action of Encode as JSON, then use this action to save as file.
Files were saved into the documents directory in the Surroga sandbox. To view these files, click the Surroga icon on the system status bar, then click Open Documents on the menu.
Read the contents of a file and assign it to a (basic) variable.
If you want to read an array or dictionary from a file, you can first read the JSON text into a basic variable and then perform the action of Text to Array or Text to Dictionary to decode from JSON.
This action access the files in the documents directory in the Surroga sandbox.
This action is used to open an application. The application can be pre-selected or specified through a variable.
Variables
A variable is a container that used to store content such as text, number, or image. Variables will make your scripts more powerful and flexible.
There are three kinds of variables are supported in Surroga. They are Variable, Array & Dictionary.
Scope
In Surroga, the scope of variables are global. Variables are shared by all scripts.
Therefore, to returns a value to its parent script, assign the value to one of variables before return, then read the value of the variable in the parent script.
Basic variables also called general-purpose variables.
They are used to store images, text and text-based values, such as:
- Integer, such as "123".
- Float, such as "0.99".
- Boolean, such as "true"
- Color, such as "#00FF00".
- 2D Coordinate, such as "(10, 20)".
- DateTime, such as "1764144367.85459"
Pasteboard
In Surroga, the pasteboard is ofen used as the special variable. Using the clipboard is more convenient than using regular variables in most cases, especially when users are developing and debugging scripts, as they can easily observe the execution status of the current step. However, for a released script, we should still avoid using the clipboard and instead use regular variables.
- The clipboard is globally shared, making it easy to interfere with each other.
- In addition, the access efficiency of the clipboard is far lower than that of variables. When an application accesses the clipboard, it needs to pass data back and forth between the system and the application, which incurs a significant overhead.
User-defined Variables
In addition to built-in variables, users can also create and name their own variables, commonly referred to as user-defined variables, custom variables or named variables.
User variables are all basic variables, accessed by name. For example,
- Assign the variable named Price to value 29.9.
- Get value of variable named Price.
Array is an ordered series of elements.
- Element can be text or text-based values.
- We can get or set the element by its index in an array.
Dictionary is a collection whose elements are key-value pairs.
- Value can be text or text-based values.
- We can use a text-based key to retrieve the corresponding value, which can be any text-based object.
Examples
Let's discusses some common approaches to script writing.
Suppose there is an application scenario that monitors a certain area on the screen. If this area changes to the preset image A, the custom key '1' on the keyboard is pressed. If this area changes to the preset image B, the custom key '2' on the keyboard is pressed.
The above requirements can refer to the following logical flow.
Set the script to an infinite loop, which contains the following process:
- Capture a specified area of the screen
- If it's similar to image A, then press button 1.
- If it's similar to image B, then press button 2.
Among them, image A and B can be obtained by taking screenshots.
The above script is simple enough and very suitable for beginners. However, it hides too many details. If the execution result is not as expected, it is difficult to observe which step went wrong. To address this, we can try to break down the steps and rewrite them in the following form:
- Capture a specified area of the screen, output to pasteboard
- Calculate the similarity between the image in the clipboard and image A, and output it to variable 1.
- Calculate the similarity between the image in the clipboard and image B, and output it to variable 2.
- If variable 1 is greater than the specified value, then press button 1.
- If variable 2 is greater than the specified value, then press button 2.
Then, enable expert mode, open the log interface in the right sidebar, run the script, and observe the execution process.
How can I copy specific text into the text input box of another application window?
First, copy the specified text to the clipboard. Then open another application and focus on the text input box. Finally, use the paste action. The steps are as follows:
- Insert action: Text > Assign, as Pasteboard = "My Text".
- Insert action: App > Open, as Open App "MyApp".
- Insert action: Keyboard > Press ⌘V.
Log
There are two types of logs in the Surroga: memory logs and file logs.
Memory Log
- Displayed in the right sidebar.
- Lost after Surroga terminated
- Limit on the number of entries and will rotate automatically.
File Log
- Open Settings > Log > File Log, turn on/off the file log and view its storage usage. Click the View button to see the log.
- Persistent even if Surroga terminated
- Capacity is only limited by the available Mac storage space.
Performance
Disable Logs
Disable Log and File Log to improve the script performance.
Optimize for Screenshot
There are some tips to optimize the performance when capture screen content.
- Choice optimized for Performance instead of Quality.
- Reduce the size of the target window as possible. The best way is expose the target region to capture only.
- Output the captured image to Variables instead of Pasteboard.
Optimize for Recognizing
There are some tips to optimize the performance of recognizing content. For example, when recognize text, click/move/drag on text, recognize barcodes or qrcodes.
- Disable Build Analysis Image option. This option is used to analyze the execution of script, but it cost heavy.
Find Subimage or Color
When searching for sub-images or colors in a window or picture, try to output only the first result instead of the best result, all results, or the top 10 results, in order to avoid significant overhead.
'App Nap' is a feature in macOS that puts an application into a 'sleep lightly' state when it is running in the background and not visible, especially after a long period of inactivity with that application. This reduces CPU usage and resource consumption. The mechanism is designed to save power and improve system performance. At the same time, it can also enhance the performance of foreground applications, thereby optimizing the user experience for active apps.
However, since Surroga needs to run in the background with high performance for long periods (for example, monitoring the content changes of a window in real-time), 'App Nap' feature can also severely reduce Surroga's performance. Therefore, we must prevent it from entering a nap state.
To do this, we can minimize the Surroga window (rather than just letting it be covered by other windows).
Expert Mode
Surroga supports two work modes.
- Beginner: Essential Features. It is recommended for beginners. It's the default mode.
- Expert: All Features, with deep and precise controls, steep difficulty curve, highly challenging. It is recommended for experienced experts.
Open Surroga > Settings > Advanced, switch between Beginner and Expert.
Multi-Display
Surroga is designed to work on the main screen.
Although Surroga does not support multiple displays, it works well in most cases, especially when the target object is draw on the primary display, or when using window-positioning instead of screen-positioning. If you suspect that multi-display mode is interfering with Surroga's ability to locate target objects on the screen, you can try removing the secondary display.
Background Task
Surroga is designed to simulate foreground manual operations, so it does not support running in the background, nor does it transmit network data.
In-App Purchases
Paid features include:
- Variables
- Calculate
- Conditions
- Control Flow(IF/ELSE/RETURN, etc.) & Error Handling
- Export & Import
- Recording as Script
- Advanced Parameters for Actions
For example, when insert a new action into script, some actions in the candidate action list are disabled for picking. It means these actions are paid features. They will be unlocked after payment.
Non-Consumable vs. Subscription
Monthly-Subscription, Yearly-Subscription and Pro shared the same features but payment. All of three unlock all features. Subscriptions are for monthly or yearly. Pro is lifetime.
Still locked after payment?
Surroga embedded a built-in mini shop provided by Apple. This mini shop is fully maintained by Apple. If the payment was completed, Apple should tell the application on network available. But, if Apple is not, for network issues or AppStore backend malfunctions.
There are some sugguestions below:
- Check your internet connection.
- Restart Surroga and try again when network available.
- Click the Restore button in the built-in shop in application.
Privacy & Security
Surroga can perform lots of tasks of applications and control your Mac. As such, it needs your security permissions to do these jobs for you. These permissions may vary according to different macOS versions. In most cases, the app will need ask for these permissions at the Privacy & Security at the System Settings.
As shown in the above figure, if both items in the image are displayed in green, it indicates that this application has been granted system authorization.
Accessibility
Allows control of your mouse and keyboard, and monitoring your mouse and keystrokes. It is required for following app features:
- Automate a keystorke
- Automate a mouse movement/clicking/dragging
Screen Recording
Allows capture the screen. It is used for following app features:
- Automate to capture the user-specified window/screen.
- Automate to text recognize on the user-specifed window/screen.
- Automate to click the user-specified text/image on the user-specifed window/screen.
- Automate to search the user-specified window/screen for the user-specified image/text/color.
Privacy Statement
Data Not Collected.
The developer does not collect any data from this app.
Which app features will use screen recording?
- Automate to capture the user-specified window/screen.
- Automate to text recognize on the user-specifed window/screen.
- Automate to click the user-specified text/image on the user-specifed window/screen.
- Automate to search the user-specified window/screen for the user-specified image/text/color.
Accessibility and Screen Recording permissions are needed to implement the essensial features of this app. The storken keys or screenshots are automated by the user's own, and never send to the developer. Please visit Privacy & Security for more infomation.
What data does your app collect via screen recording?
The data, user-specified image or text, was recorded by the user's own willing. No data was collected by this app or the developer.
For what purposes are you collecting this information? What plan for uses of this data.
No data was collected by the developer. No plan to use.
Will the data be shared with any third parties?
No.
Where will this information be stored?
This information was stored in device RAM. It will lost if app quit, or device shutdown/reboot. If the users choice output the data to system pasteboard, then they can visit macOS system instead of this app.