Archive | Testing RSS for this section

On Testing

What follows is my personal journey as a tester over the past year. Some of the success, hurt and frustration I’ve had is presented in a hopefully constructive manner. The wording on this is less conversational, yet I’m hoping with those who read it it does start a conversation in some way and also reminds testers that we have value and an obligation to those who want to learn about testing.

I always feel fortunate to be a part of companies who are producing software and I couldn’t imagine doing anything else. I am working with interesting, intelligent people and tackling problems that solve problems for other people. We are solving these problems using computers and code that create solutions that weren’t available to the majority of people twenty years ago. When I reflect on it, I’m amazed at the opportunities I am being offered.

The changes in the industry have been gigantic within the past few years, you can see them like tidal leviathans on the horizon. The learning and technical tools I have had access to are a result of those movements: Applications available from GitHub, public infrastructure from Amazon Web Services, a library of knowledge from experts and knowledge repositories easily accessible through Google searches. Last year I read twenty-two books for my career, ranging from test automation to Continuous Integration to C# and I have many more waiting to be read. The vast access to jumpstart skills sparks a fire I want to tend to and steadily grow until I can light fires for others and start them on their journey into the uncertainty of solving complex technical problems.

What does this have to do with testing software? The above prose could be for almost anyone who releases software. This is the point, the basis of what I want for testers. I want them to be equals alongside the artists and engineers, dancing around the fire with their teammates, sharing in the same danger and excitement.

People who work with testers want it too. As we shift to more frequent and adaptive releases and our projects involve more disciplines, testing raises in its importance. The activity of testing is not valuable, otherwise we would take more time for more elaborate and time-consuming tests. It is because of our skill and the value of what testers do when they apply them. For contrast, if the code developers wrote was the most valuable thing they could do, they would focus on producing more lines of code. Testing skills are discrete and valuable to a team, and it’s a full-on weeping shame that testers don’t appreciate or exercise them enough. Do you see the look of wonder on Aragorn’s face as he accepts Andúril and ultimately his responsibilities in the oncoming conflict? We should value ourselves and our role as much. Testers do not assure quality, yet we can apply tools like empirical thinking experiments, exploratory testing and general system thinking to help determine the solution’s qualities and readiness as part of a greater whole. Who doesn’t want mindful articulate thinking applied to their work as software development leans more towards business opportunities and less towards technical requirements? There is great concern and risk in releasing software on the Internet for a company. Companies want to invest money to have a more successful acceptance from those paying for it and those using it.

Aragorn holding Anduril
Aragon accepting Andúril  is a metaphor of what our acceptance
of our responsibilities 
as testers to our teams should be.

Developers who refuse to test are becoming scarcer as testing becomes a whole-team activity and the same is true for testers who refuse to code. Testing and coding are not exclusive skills and they can be learned and applied by the same person. Two applications of testing I have started to learn within the past year, unit tests and integration tests, still employ many of what I think of as the valuable thinking skills I’ve already mentioned. When we treat testing as a specialization instead of our role we can disrupt the pattern of segregation and share our testing skills to create a healthier team.

The companies, the methods, the tools, even the positions we’re hired for are always changing and until that itself changes, refusal to change and work as part of a team and adapt to the changes interferes with the quality of the software and the value it provides to its customers.

Edit: Replaced some words for general correctness.


Writing a UI automation framework with C#, part one

I spent some time earlier this year digging into UI automation, specifically an implementation a peer had started using C#, Selenium WebDriver and NUnit. At the time there was an ask for us to implement a sanity-level test of our team’s product and it seemed like the right time to become more familiar with the concepts around working with WebDriver.

What I found in the existing framework was an abstraction layer for WebDriver, and other objects to provide functionality for making testing more presentable to testers that had some familiarity with object-oriented concepts. There was also logic in the test fixture setup and teardown (a concept introduced from the NUnit test framework) to handle loading language-specific variables. These were all good features making a tool we could use in a variety of ways, and they aligned with what I was reading online, and we’re still using them today to develop our tests.

When I started focusing more on the UI automation we were doing, I started writing my own UI automation framework. My approach was to make something comparable based on what I was learning about C# and WebDriver, without using any reference material. Within a couple days I had something running, and in another day I had something that ran on Windows and Linux.

This is evidently a long preamble so that when I start explaining my design I don’t sound self-aggrandizing. There is also nothing particularly special in this article that can’t be found online and written by people that are much more into this than I am.


The UI automation framework I wrote contains a WebDriver abstraction, an element locator, explicit wait conditions and objects to support the page object design pattern.

WebDriver Abstraction

In order to provide more friendly methods to some of the obscured WebDriver functions I wanted access to, I created an abstraction object that translates something I want to do into the appropriate WebDriver call. For example:

public void SetImplicitWait(int seconds)

Aside from presenting a subset of more direct and specific user interactions with the WebDriver, this object also represents actions the user wants to do. So there are methods for clicking an element or entering text into an element on the page. (It’s worth mentioning that the elements are actually located in the DOM, but that the end result is a page being rendered in a browser, so I use page because it’s more evident.)

The WebDriver interfaces are still accessible through this object as well, so it’s possible to make calls directly if it doesn’t make sense to work through this abstraction layer. One example of this from my experiences so far, are for selecting a group of elements instead of one specific element using a FindElements method call.

Element Locator

Specifying elements on the page is a common action when working with UI automation using the WebDriver. So that a single object with readable methods would provide a way of returning a By object (used for working with other WebDriver methods, like the very important FindElement) I created a static object that did this. Here is an example of a call:

public static By ElementByValue(string value)
return By.XPath(String.Format("//*[text()='{0}']", value));

Others are more direct, taking a name of an element and doing a more direct lookup, for example.

Explicit Waits

WebDriver has the concept of implicit and explicit waits. In C#, it has a specific way of providing a wait condition through a lambda statement. I’ve found there are some common situations where you want to be able to wait, and that some of these are when you’re interacting with the WebDriver, but that this is also relevant when working with pages or tests. I made this functionality visible to any object wanting access to it.

An example of providing a wait for an element to become visible looks like:

public static void UntilElementIsVisible(Driver driver, By locator, int timeout)
WebDriverWait wait = new WebDriverWait(driver.webdriver, TimeSpan.FromSeconds(timeout));
wait.Until((d) => { return d.FindElement(locator).Displayed; });

Page Object Pattern Objects

From the different patterns I’ve seen, and from the different ways I’ve written Selenium scripts, for comprehensive functional automated UI testing, I like the page object pattern. Although I’m not as strict about how it’s generated or used, I found it useful to provide a base object with a few methods for what I would expect someone writing a test to want from a page they were specifying. These methods are for determining if an element is visible to a user or if an element has a specific value. An example of this would be:

public bool ElementHasValue(By locator, string value)
IWebElement element = driver.webdriver.FindElement(locator);
return (element.Text == value);

This also highlights another point in my overall design, to leave user actions to the Driver object and page validation actions to the base Page object.

In Closing

In writing this I hope it helps others who were at the spot I was at earlier this year, putting together the pieces of UI automation, in getting started on their journey.

For those interested, I’ve made this framework available on GitHub. I called it Kitsune, because right now it works with Firefox and I’m a fan of Japanese folklore and games which promote that rich cultural tradition. It has no relation to the Mozilla Foundation’s Kitsune project and they do two very different things.

How Google Tests Software by James A. Whittaker, Jason Arbon and Jeff Carollo


When I was on vacation towards the end of last year, I wanted to read a book that would help me along the path to become a better software tester. With that in mind, I chose How Google Tests Software and learned how a giant search company keeps software testing in-line with development practices.

For content, the book is divided into five sections: Introduction to Google Software Testing, The Software Engineer in Test, The Test Engineer, The Test Manager and Improving How Google Tests Software. This book serves two audiences: Those looking to learn from the test processes that define how Google tests its software, and to Nooglers (new Google employees) who need a launch point for learning about testing activities. Although the first four chapters serve both groups, the last section is more angled at the Nooglers. There are appendices at the back that show sample documentation for the described test processes.

This book approaches the major themes and concepts of testing activities at Google, and occasionally dips into more detailed information, such as tools that were developed or specifics of a task, like a brief glimpse into repo branching or the workflow of a Software Engineer in Test. What I learned from it is

  • The value of manual testers to solve specific problems
  • How prevalent coding is in all testing roles
  • Where automated tests can validate different parts of a system
  • Where a tester can develop tools and frameworks that help developers in their testing activities
  • How no one has it right and we all struggle with different problems

I’d recommend that testers read this book to get a look at how another company tests software. I learned things from it that I want to incorporate in my work and chances are others will too.

Agile Testing by Lisa Crispin and Janet Gregory

Agile Testing introduces a software tester to working in an agile environment through introductory and intermediate topics, continuously explaining how they can work as a member of an agile team.  It shifted my understanding of what testers are and how the reader can contribute to software development projects.

The book provides exposure to a variety of topics, from culture to project management to coding.  Some of these sections are covered more in depth, such as the Agile Testing Quadrants.  The explanation of this concept and how it can help to define the testing that would provide value to the technical (team) and business (user) stakeholders was very valuable, and led me to share this information with our team during an informal lunch and learn presentation.  Other areas that helped me to think of my job in a different way were the chapters on agile teams and the relationship between coding and testing.

Other sections felt less useful, such as the sections on automation and culture.  These topics seemed to provide apparent information and didn’t go into the depth other areas did.  From this I have selected other books that tackle these topics specifically, so that I can go into greater depth on these topics.

A feature of this book worth noting is the the extensive examples, either from their own experiences or others also practicing agile testing methods.  As I read this book, I came to increasingly appreciate these examples.  They either provided affirmation when I already knew the concepts I was reading or showed an implementation when I needed to improve my understanding of a topic.  I never missed an example, they were that valuable to me.

I would recommend this book to any person looking to understand how testing provides value on an agile team and need examples of what that can include.  It is at the least an introduction to this type of role, but in my experience it went beyond and changed how I approach testing.

%d bloggers like this: