Automation is probably most underused thing in regular desktop computing. Few techniques deliver that much benefits as simplifying and streamlining repetitive tasks – exactly what computers are meant for. Where user sees windows, icons and mouse cursors – computer sees mess of numbers and labels. Mess that you must usually learn to use languages like AutoIt and biggest turn off.
Sikuli takes drastically different approach by implementing visuals in scripting. It cares how button looks, not how it is defined at computer’s level.
What it does
Sikuli allows to build scripts that use captured parts of screen for active areas. You simply show it what button you want to press looks like. Or shortcut. Or link. It doesn’t matter what it is, only how it looks.
For example CCleaner lacks (on principle) command line support to automate registry clean and I wanted to script it. In Sikuli this would be:
Graphical areas are screen-captured from interface or imported from images. You can test image search (click the defined graphics) and regulate how similar area must be to match.
Scripts can be saved as folder with code+images or executable files, that can be launched if associated with Sikuli.
Strong features
Sikuli is currently in beta and far from full-featured scripting language. But there is enough in place to see power of concept. It makes automation both graphic and universal – features it desperately needs to have a shot at mainstream adoption.
Downsides
Documentation is zilch. App itself has none and site only has very basic description of available functions.
It took me some time to even figure out that it actually uses Python syntax. And since I don’t have a clue about Python I had to look that up for my rather simple script.
It works good for click-click-click, but anything more complex and you are neck-deep in exactly those intrinsic programming details that Sikuli is supposed to eliminate from process.
Overall
Amazing tool that is sadly not ready for serious usage yet. For me it is firmly one of the most anticipated pieces of software ever.
Home&download http://groups.csail.mit.edu/uid/sikuli/index.shtml