操作系统指令

注意: 这部分 api 是操作系统原生 api, 浏览器内部场景无法使用。操作软件菜单, 系统鼠标事件等

获得屏幕和进程窗口信息

def screenInfo(): util.List[util.Map[String, Object]]def windowInfo(title: String): util.List[util.Map[String, Object]]

激活窗口

激活后后续的操作都是相对于激活窗口的

def activate(pid: Long): Systemdef activate(pid: Long, idx: Int): Systemdef activate(title: String): Systemdef activate(title: String, idx: Int): Systemdef inactivate(): System

截图-系统屏幕

def screenshot(): Array[Byte]def screenshot(x: Int, y: Int, width: Int, height: Int): Array[Byte]

屏幕裁剪

def crop(): ImageElementdef crop(locator: Object): ImageElementdef crop(x: Object, y: Object): ImageElementdef crop(x: Object, y: Object, width: Object, height: Object): ImageElement

鼠标事件

def move(x: Int, y: Int): System // 移动到屏幕坐标位置def click(): System  // 鼠标左键def midClick(): System // 中间def rightClick(): System // 右键def doubleClick(): System // 双击左键def click(x: Int, y: Int): System // 移动鼠标并左键def press(): System // 左键按下def release(): System // 左键释放def input(value: String): System // 键盘输入def delay(millis: Int): Systemdef highlight(x: Int, y: Int, width: Int, height: Int): System // 高亮屏幕区域

示例

* print sys.screenInfo()* sys.screenshot()* print sys.windowInfo('idea')* sys.activate('idea', 0)* sys.screenshot()* sys.move(400, 500).rightClick()* sys.move(500, 500).click()* sys.crop(200, 200).highlight().screenshot()* sys.crop(0.5).highlight().screenshot()* print sys.crop(0.5).ocrExtract()

* sys.crop().detect()* sys.crop().detect('gftt', {minDistance:3, useHarris:false})* sys.crop().detect('mser', {'minArea':100,'maxArea':800})* sys.crop().detect('harris')* sys.crop().detect('fast', {threshold:10})* sys.crop().detect('surf', {hessianThreshold:250,extended:true})* sys.crop().detect('sift', {nFeatures:0,nOctaveLayers:3})* sys.crop().detect('morph', {shape:'ellipse',width:16,height:4})

# 屏幕 (50,20) 宽高 (100,200) 的区域找到`关于`菜单, 点击* sys.crop(50,20,100,200).find('关于').click()