SilverLight tests + PowerShell = Automated tests in MSBuild
In this post I will only discuss the way to configure SilverLight unit tests to run completely automated from MSBuild, so that it can also be used as part of continuous integration setup.
SilverLight tests must run within an actual SilverLight plugin rather then .NET Framework. Fortunately, Microsoft provided a test framework and source code for the both SilverLight controls and unit tests.
It is great to get support for TDD for such a new technology as SilverLight 2.0 right from the start. However, it is not possible to run SL tests from MSBuild out of the box. But do not panic, there is an easy solution to thid using PowerShell. PowerShell allows you to use COM objects and fortunately IE comes with a COM API that we can take advantage of. Below is a PowerShell script that creates and redirects IE browser to the SL test page, re-formats the test result output and saves it as an html file. It also returns an exit code > 0 in case of test failures.
param($testURL, $outFile=”TestReport.html”, $visible=$true)
#Instanciate IE, set Visible mode (tests that interact with desktop will require $true to pass
#then navigate to the test page
$ie = new-object -com “InternetExplorer.Application”
$ie.Visible = $visible
$ie.Navigate($testURL)
#poll the content until test execution is complete.
#Note: using $ie.Busy won’t work, because it does not detect that SL application is still running.
#Furthermore, the SL application is running until you close the browser, even after all the tests complete
while (!($ie.Document.body.innerHTML -match “\[ Close \]”))
{
[System.Threading.Thread]::Sleep(100)
}
#Once the execution is complete, we can check the results
#and, thanks to artist rendition by D. Finke, display some shining colors!
$failure = $ie.Document.title -match “(\d*) failures”
if($failure) {Write-Host -NoNewline -ForegroundColor “red” “Failed $matches[1]”}
else {Write-Host -NoNewline -ForegroundColor “green” “Pass “}
#Heavy RegEx stuff to reformat the body HTML.
#The test results in the SL test framework are displayed as a narrow column on the right with some
#error messages wrapped into context pop-up or limited in width.
$regOpt = [System.Text.RegularExpressions.RegexOptions]
[regex]::Replace($ie.Document.body.innerHTML, ‘^.*<DIV style=”RIGHT.*?</DIV></DIV>’, “”, $regOpt::Singleline) -ireplace “width: 324px”, “” -ireplace ‘<SPAN title=\”(.*?)\”.*?\(Mouse over for assert trace\)</SPAN>’, ‘$1′ -ireplace ‘</DIV>\w*$’, “” > “$outFile”
#return error code in case of failure so that MSBuild and other tools can detect it.
Write-Host $testUrl
$ie.Quit()
if ($failure) { exit 1 }
Now, after we have out PowerShell script we can call it from MSBuild and have a completely automated SL unit test setup.
Note: If you will be running it from a service, i.e. TeamCity build agent, you should enable the service to interact with desktop.
Below is a snippet from MSBuild configuration file that calls the SL unit tests via PowerShell. Also, you will need to set PowerShell execution policy to allow the script to run using Set-ExecutionPolicy cmdlet. Check google for available policies.
<Target Name=”RunUnitTests”>
<Exec IgnoreExitCode=”True” Command=’Powershell .\GetSLTestReport.ps1 “[Your Location]\TestPage.html” ControlTestResults.html’>
<Output TaskParameter=”ExitCode” PropertyName=”ControlTestExitCode”/>
</Exec>
<Error Condition=”$(ControlTestExitCode) > ‘0′” Text=”SL Unit Tests run failed with code $(ControlTestExitCode)”/>
</Target>
Note: I am using IgnoreExitCode option to allow other tests/tasks to run in case ControlTestResults.html fails. If you do not need this, you can remove IgnoreExitCode and <Error> tag.
=>
<Target Name=”RunUnitTests”>
<Exec IgnoreExitCode=”True” Command=’Powershell .\GetSLTestReport.ps1 “[Your Location]\TestPage.html” ControlTestResults.html’ />
</Target>
