Selecting from a dropdown generically with Capybara

For my current project I am developing some smoke tests using Capybara. For smoke tests you want to get from the start to the end of a process without caring too much about the specifics of each page. Additionally, we want to  avoid hardcoding in the tests any data that changes. Otherwise data changes may lead to a misleading test failure (a false positive). In this project there are several dropdowns that contain lists of suppliers, energy plans,  etc. For the smoke test we just want to select a valid option and then move on to the next page.

To help with this I wrote some generic code to choose the second item in a select list – the first item usually being a “please select” message. To do this I use the XPath support in the Capybara API:

def select_second_option(id)
  second_option_xpath = "//*[@id='#{id}']/option[2]"
  second_option = find(:xpath, second_option_xpath).text
  select(second_option, :from => id)
end

The code uses this XPath: //*[@id='#{id}']/option[2] to select the text value of the second option. The standard Capybara API is then used to select that value in the dropdown.

Advertisements

Cure deployment pain with Environment tests

You have just finished a cool new feature for your application. Everything works hunky dory on your machine. You are super confident everything is going to work perfectly when you deploy because you have lots of automated tests – unit tests, integration tests and even acceptance tests. You install the application on a user’s machine and they use it for about 30 seconds before an error pops up. Woops, they are denied access to the database. After 10 minutes fiddling around with database permissions you assure them that everything will work fine now.

Five minutes later the user is back at your desk. When they look up a postal address in the application they get some error about “Cannot connect to address lookup web service….”. After some investigation you realise that there is a typo in the configuration file. After 30 minutes they finally have a running application. Sound familiar?

The problem is that applications depend on many small things being configured correctly on both the machine they run on and the services they connect to. The database connection strings and web service URLs in the application’s configuration file must be right. The user must have the correct permissions for any databases or network folders the application requires. These problem are hard to avoid if not inevitable. Given that fact, the best solution is to built system that fail fast, diagnose problems and tell you up front what is wrong and even how to fix it. That way you can avoid losing hours diagnosing the root cause of a problem.

Environment tests to the rescue

To find these problem we write Environment tests a much under utilised technique I first read about in Jeremy Miller’s blog post Environment Tests and Self-Diagnosing Configuration. Environment tests are small focused pieces of code that are packaged with the application and verify access to a resource (such as a database, folder or WCF service). Environment tests are sometimes called smoke tests, canary tests, self tests or diagnostic tests.

I have implemented a small lightweight framework to write these tests. Each test implements this interface:
public interface IEnvironmentTest
{
    void Test(ILogger logger);
}

Here is an example of a database connectivity test:

  public class DatabaseConnectionTest : IEnvironmentTest
    {
        public void Test(ILogger logger)
        {
            logger.LogInformation("Testing database connection");
            for (int i = 0; i < ConfigurationManager.ConnectionStrings.Count; i++)
            {
                var connectionStringSetting = ConfigurationManager.ConnectionStrings[i];
                if (connectionStringSetting.Name == "LocalSqlServer")
                {
                    continue;
                }

                var connectionMessage = "Connecting to database " + connectionStringSetting.Name;
                logger.LogInformation(connectionMessage);
                using (var conn = new SqlConnection(connectionStringSetting.ConnectionString))
                {
                    conn.Open();
                }
                logger.LogInformation("Connected successfully");
            }
        }
    }

The code is quite simple. It loops through each connection string in the application’s configuration file (skipping LocalSqlServer). It then logs the name of each database, attempts to connect to the database and logs if the connection was successful. It is quite important to log exactly what is happening as tests results will be used to diagnose problems. If the connection fails an exception is thrown by the Open method of the SqlConnection object. This exception is then caught and logged by the EnvironmentTestRunner object:

public class EnvironmentTestRunner
    {
        private readonly IEnumerable tests;

        public EnvironmentTestRunner(IEnumerable tests)
        {
            this.tests = tests;
        }

        public EnvironmentTestResult Run()
        {
            var passed = true;
            var logger = new StringLogger();
            logger.LogInformation("Starting environment tests");
            foreach (var test in tests)
            {
                try
                {
                    test.Test(logger);
                }
                catch (Exception exception)
                {
                    passed = false;
                    logger.LogInformation(exception.ToString());
                }
            }
            return new EnvironmentTestResult(passed, logger.Log);
        }
    }

The EnvironmentTestRunner object executes the set of environment tests that are passed to it. If any test fails it captures the exception and adds the exception details to a test log. It then continues on to the next test. When all the tests have finished it returns an EnvironmentTestResult (either passed or failed) with a test log with the output of the tests that have run:

    public class EnvironmentTestResult
    {
        public EnvironmentTestResult(bool passed, string log)
        {
            Ok = passed;
            Log = log;
        }

        public bool Ok { get; private set; }
        public string Log { get; private set; }
    }

Running the tests

Where and how the tests run depends on the type of application you are writing. For my current project, a windows call centre application, the environment tests run on application startup. If any of the tests fail the user is shown a dialog box with the environment test log. This allows the user to let us know straight away if something is wrong. The test log provides the development team with detailed information of where and what the problem is.

For WCF services we added a service endpoint called Status that runs the tests and returns the result. This has proven useful as services have no user interface making it tricky to diagnose problems. Often in the past we resorted to writing one-off integration tests to investigate a problem in a production environment. Now we have a Cucumber test running on our Continuous Integration server that calls the status method on the service and reports problems. We are also investigating integrating the status checking with our service monitoring tools.

For web applications we have a page which runs the tests. As these pages are externally visible, the page does not report any error information in the user interface as this would give away sensitive information. Instead, the test log is saved to our error log. The page display a pass/fail message and returns an HTTP error code. A Cucumber tests running on our Continuous Integration server loads the page and reports a failure if an HTTP error code is returned.

Making a problem go away and stay away with environment tests

In recent weeks we had a problem with users not having write access to the security certificate store on their machines. This was a time-consuming problem to nail down. The first thing I did was write a test that replicated the specific problem. This was useful as the error message that the problem caused was quite obscure. Now when the application runs it will immediately detect if the problem is present, what the problem is, how to resolve it and give a URL with more information. The problem may happen again in future as we had to change permission on the user’s machine to fix the issue.  We can be confident however that the application will tell us the problem and not have to rely on external documentation, code comments or our memory to fix the problem.