I had a great idea this morning... and then realized that there was an open source project already doing it.
I thought: instead of using specific testing frameworks for each UI technology I have (a tool for the web, a tool for SWT UIs, etc) what about a tool that could recognize the text on screen and click or type in the proper places. The same technology for everything type of UI, and something that doesn't rely on pixel perfect positioning to work! Turns out (surprise!) I'm not the first one that had the idea, and looking around I stumbled on Project Sikuli. This looks great: it uses images to find "interesting" parts of your UI and there's also support for text recognition. So potentially you could say "click on the button that says 'Submit'"! It integrates with Java code so you can easily have JUnit tests using it.
Unfortunately, for my own purposes I couldn't get very far easily because of lack of support for transparent images (or, more precisely, it takes into account transparent areas for images, so it doesn't recognize an image if the background changes, which happens if your desktop theme changes, if you use the same icon in different contexts in your UI, etc). But it still looks like a promising tool!
6 comments:
Hi ,
If you are looking at an end to end test solution with OCR functionality for test automation across any OS any device . Try eggPlant from TestPlant.
http://www.testplant.com/blog/2011/07/22/eggplant-with-ocr/
http://www.youtube.com/user/TestPlantQAeggplant?feature=mhee#p/a/u/1/5XUa0m7fzHU
But it is a commercial tool - with complete support.
Try it to see the utility.
hii sir im new in sikuli so can u explain me integrating sikuli with java
It's true that the documentation talks about Python, but you can as easily use Java. Just include all the Sikuli jars in your project. Then :
Screen s = new Screen();
and you can use the API of Screen (wait, click) to manipulate your application. What I've done is put all this code in a JUnit test, that starts my application then tests it via Sikuli.
Hello Sir..
I have add the sikuli jar file in the class path of system variable C:\Program Files\Sikuli X
and after that i add path to sikuli\libs i.e. C:\Program Files\Sikuli X\libs in the path of system variable
and path to java\jre6\bin i.e. C:\Program Files\Java\jre6\bin in the path of system variable
and in the command line i write:- java -jar "C:\Program Files\Sikuli X
\sikuli-script.jar" imgs2.sikuli // imgs2.sikuli is the name of the
folder where i capture screen shot of the start button//
but it doesnt run in the command prompt..
and i write a program in java but i also show some error
package defaultt;
import edu.mit.csail.uid.FindFailed;
public class anu {
public static void main(String[] args) throws FindFailed {
Screen s = new Screen();
s.click("imgs2/start.png", 0); // this command use for click on the start button//
s.wait("imgs2/AnuragVM4Int.png"); // this command use for wait to open the start menu/
s.find("imgs2/Run.png",0); // find the run button //
s.click("imgs2/Run-1", 0); // click on run //
}
}
now can u help me to tackle this problem...
if i made any mistake in command prompt, adding jar file and java file in system variable and in writting java program then plss guide me
Er, you don't give the error message, so I can't guess what the problem is.
Anyway, posting on my blog "project Sikuli is a great idea" doesn't make me official tech support for it. Please read the information on the Sikuli web site, and ask questions on the proper channel, which I believe is https://answers.launchpad.net/sikuli.
Good luck!
Post a Comment