Asynchronous Unit Testing in Xcode 6

Last year I described a method to implement asynchronous unit testing in Xcode 5.

Let’s remind ourselves of the problem with asynchronous unit testing. Many APIs on the iOS platform themselves are asynchronous. They have use callback invocations to signal when they’re completed, and these may run in different queues. They may make network requests or write to the local file system. These can be time-consuming tasks that need to run in the background. This creates a problem because tests themselves run synchronously. So our tests need to wait until they are notified of when the running task has completed.

I proposed a method that entailed setting a boolean flag in the unit test and looping in a while() loop until the flag was set to false, allowing the test to complete properly. This method worked most of the time but I have never been happy with it, regarding it as a bit of a kludge. In that blog post I concluded:

I still have my reservations about this technique, and I’m still looking for the perfect solution for asynchronous unit testing in Xcode. You would think that Apple might have provided a solution in XCTest, perhaps similar to the implementation in GHUnit.

Here’s what the Objective-C version of a bare bones example asynchronous unit test in Xcode 5 using the old method looks like:

In fact, because I was re-using the same pattern in many of my tests, I converted parts of it to a Macro that had to be included in each header file. Also, I noted that under some conditions the test didn’t complete properly.

Well, the good news is that, less than a year later, Apple have delivered a means to implement asynchronous unit tests in an intelligent and offcially supported way. Furthermore, not only have they given us a new version of Xcode 6 (still in beta at the time of writing) with this new unit testing framework, but they have also delivered a brand new programming language, Swift. I’ve spent some time over the last few weeks converting a hefty chunk of Objective-C code to Swift, and in converting my Unit Tests to the XCTest framework I implemented Apple’s new methods for asynchronous unit testing. From now on, all of my iOS coding is going to be done in Swift, so the examples below will be in Swift, too.

So, how does it work? In Xcode 6 Apple have added some extensions to the XCTestCase class, and I’m going to focus on two of them:

There’s also a new class, XCTestExpectation which has one method:

Basically, you declare an “expectation” in your unit test, and loop in a wait loop waiting for the expectation to be fulfilled in your code. It’s the same pattern as before, but with more options. Here’s the old Objective-C code converted to Swift using the new framework:

In line 6 we instantiate a new instance of XCTestExpectation, named readyExpectation. We give it a simple description for convenience, “ready”. This will be displayed in the test log to help diagnose failures. It is also possible to set more than one expectation as a condition. Then in line 9 we make the call to the code that needs to be tested. In the completion handler, after making our tests, we call the method fulfill() on the expectation. This is equivalent to setting the flag to false in our earlier Objective-C implementation.

The last block of clode starting at line 18 runs the run loop while handling events until all expectations are fulfilled or the timeout is reached. I set the timeout to 5 seconds to be on the safe side.

And that’s about it. There’s more you can do with the new additions to the unit test framework, such as key-value observing, and performance metrics, but the above should be sufficient to get going. Finally, we have a proper framework for Asynchronous Unit Testing in Xcode!

Eclipse and Asynchronous Unit Testing

I’ve been retro-fitting hundreds of JUnit tests to Archi‘s code base. It’s a Zen meditation, as the process of devising the unit tests ensures that I re-engage with the code, clean things up, refactor here and there, and finally understand what that nifty algorithm does that I wrote 3 years ago. Like children not eating their greens, no-one wants to write unit tests. But it’s a valuable thing to do and rewarding, especially when you see the green bar in Eclipse:


Perhaps the process of testing and being rewarded by the green bar is an example of “gamification”? Maybe, but I’m not interested in vacuous neologisms, I’d rather just get stuff done.

Some tests that I wrote were failing. The code that these tests was calling depended on events occurring in the UI of the application, such as an editor window opening. The tests were finishing before the editor window opened. Not sure what to do about this I went for a jog to have a think. And then I remembered that I already had the answer in one of my own posts, Xcode and Asynchronous Unit Testing. This has turned out to be quite a popular post in the Xcode and Objective-C community, solving a common problem. So I wondered if Eclipse would be happy with a similar pattern. I’m never sure what’s going to happen when I start playing with threads and dispatch queues.

Here’s my solution, an abstract Java class called “AsyncTestRunner”.

This class implements Runnable which has one method, run(). This run() method will be called by the current Display instance asynchronously in this line:

Meanwhile the read and dispatch process will go off and do its thing in the event loop:

This ensures that UI tasks can be completed.

Here’s an example of how we use it:

Here, we create a new anonymous instance of AsyncTestRunner and implement run() with the testing code and then call its start() method. Be sure to call super() in the run method to invoke stop() which sets a boolean flag to false which breaks the event loop in the start() method.

The class may seem verbose with its stop() method, you could simply directly set the waiting flag to false, but I wrote it like this so it can be sub-classed and to make it obvious what’s going on. I also like the idea of a run() method consisting of one stop() method.

Very Zen.

So far it works in both Eclipse 3.8.2 and Eclipse 4.4. I may find a gotcha further down the line, and I may refine the class in the future, but so far it’s looking good. If it can be improved I’d be glad to hear about it.

Draw a line in the sand

I’m reading the book “Rework” by Jason Fried and David Heinemeier Hansson. It’s the right book for me to read right now. I’ve been thinking that as David is to Goliath, so is Archi to other tools. And here’s a quote from the book that resonates with that thought:

As you get going, keep in mind why you’re doing what you’re doing. Great businesses have a point of view, not just a product or service. You have to believe in something. You need to have a backbone. You need to know what you’re willing to fight for. And then you need to show the world.

A strong stand is how you attract superfans. They point to you and defend you. And they spread the word further, wider, and more passionately than any advertising could.

Strong opinions aren’t free. You’ll turn some people off. They’ll accuse you of being arrogant and aloof. That’s life. For everyone who loves you, there will be others who hate you. If no one’s upset by what you’re saying, you’re probably not pushing hard enough. (And you’re probably boring, too.)

Lots of people hate us because our products do less than the competition’s. They’re insulted when we refuse to include their pet feature. But we’re just as proud of what our products don’t do as we are of what they do.

We design them to be simple because we believe most software is too complex: too many features, too many buttons, too much confusion. So we build software that’s the opposite of that. If what we make isn’t right for everyone, that’s OK. We’re willing to lose some customers if it means that others love our products intensely. That’s our line in the sand.

Sand Line

But, of course, lines can always be re-drawn. Boundaries are constantly shifting depending on demand and opportunities. Perhaps, too, the wind will blow away the line in the sand and a new one drawn?

Zombie Organisations

Max Keiser rails against “zombie” banks, and rightly so as the economy suffers from the life-draining consequences of these wretched institutions. A zombie bank is a bank that that has an economic net worth of zero, or less than zero, but continues to operate as a result of government backings or bailouts.

But what about zombie organisations?

A zombie organisation is an organisation that also has a net worth of zero, or less than zero, since it has had all of its life-blood drained out of it and yet somehow, inexplicably, continues to walk the earth. Or, more accurately, the life-blood has left it – the good people of the organisation, the creative and honest people, have long since left to go onto better things, no longer able to endure the lack of management, and the incompetence and solipsism of the zombie organisation’s funeral directors. However, unlike the zombie banks, the zombie organisation is not being bailed out and instead has lost its source of funding, its fiscal blood supply, and yet stumbles on relentlessly, zombie-like, pursuing its mindless course.

I don’t know which is the saddest, the zombie bank or the zombie organisation. At least you can understand how the zombie bank continues in its unfortunate existence, pumped with the blood of tax-payers’ money. But how do you explain the continued existence of the zombie organisation, deprived of the blood of funding, unwanted even by its own host organisation, circling itself in its own uselessness and redundancy? Just how do you put the zombie organisation out of its misery?

Do you love your users?

Alan Rusbridger, the editor of The Guardian newspaper, was recently asked by a UK home affairs committee investigating the Snowden leaks, “Do you love this country?” This is not a question I will answer, but I do find myself asking a similar question of software developers – “Do you love your users?” Or, put another way, “Why do you hate your users?”

As the developer of Archi, the free ArchiMate modelling tool, I’ve put in many hours of my time, and much soul-searching, trying to create an application that works simply and elegantly, hiding complexity from the user and, above all, not crashing. I’ve thought deeply about what users want and what they probably don’t want, and I’ve listened to, and responded to, user feedback. So my answer to the question, “Do you love your users?” is unequivocally, “Yes!”

I’d like to illustrate this with an example of the process of thinking about usability in my software, and finish with an example of “user hatred” in another piece of software.

Show me the love…

I just implemented a new feature in Archi – to let the user define the line colour of a selected object in a diagram. This wasn’t too difficult, I just followed the standard presentation in the Properties pane. But then it got a bit more complicated when I added an additional option to derive the selected object’s line colour from its fill colour. Again, this wasn’t too hard, but it did present a usability challenge – one option negates the other. If the latter option is selected in the application’s global preferences, then any user line colours are ignored. So how would this look in the object’s Properties pane?

Archi Properties pane

Why are they disabled?

The problem is that the line colour selector buttons are always disabled. Whatever you do, they remain disabled since the global preference is set to derive line colours from the object’s fill colour. So why on earth are the buttons there? Perhaps the user needs to do something special to the selected object to enable them? Search in the user manual? Perhaps phone the help-desk? As it stands, this is a perfect example of the software designer hating their users. It’s the equivalent of inviting someone to dinner and then hiding the food in the cupboard.

So let’s show some love and try and solve the problem. Let’s kindly explain to the user that if they want to set their own line colour, they should have not been so stupid to have set it in the application’s preferences in the first place:

Archi Properties pane - better

Hmm…why should I?

Well, why should the user have to do that? They can’t even remember where the preferences are, let alone which preference to look for. Out with the manual again!

Perhaps the better solution is to not only explain why the feature is disabled, but also to offer a link that will open the preferences dialog window and take the user to the preference that will solve the problem:

Archi Properties pane - much better

A bit better, but…

Clicking the link takes the user to the Preferences dialog window:

Archi Preferences dialog

Preferences dialog – set it here

This is not a perfect solution, since the user has to think about whether to disable or enable the option. However, it’s a lot better than before. But wait…

…why not just completely hide the colour buttons in the Properties pane when the feature is not available? This way you wouldn’t even need to show an explanatory text and link. Well, I thought about this and this creates a different problem – the user might never know that the line colour feature exists. By greying out the buttons I’m providing a hint to the user that the feature is available under some circumstances. The additional explanatory text and link then takes the user to the preferences dialog so that they can see for themselves how and why this is an option – coaching and coaxing the user.

Even this solution sucks. There are just too many buttons cluttering up the screen. Back to the drawing board.

This is better:

A better solution

A better solution

Here I’ve combined all buttons into one control – the colour selector, as before, and the “Default” and “Preferences” into a drop-down menu available from the right of the selector. I even re-used the control for the Font selector. Nice.

And now for something completely different…

That is a small example of the process I go through when demonstrating the love I have for Archi’s users. But what about software developers who seem to hate their users? I give you Protégé, an open source ontology editor from Stanford University. Here’s the opening screen on Windows:

Protege - WTF?

Protege – WTF?

This just broke Software Rule #1 – Don’t Throw Shit at the User.

Ok, how about simply opening a file in the current editor window? Here’s the result:

Protégé - oh dear...

Protégé – oh dear…

Protégé also has a menu item called “Refresh User Interface“. Really? You don’t get a dog and bark yourself. Come on, this is supposed to be Stanford Frickin’ University.

You know what? I think the Protégé developers hate their users.

I finally found the long lost bug in Archi

I finally traced and fixed a nagging bug in Archi that can lead to orphaned diagram elements or, put another way, a corrupt model file. This was an issue that had occurred for a small number of users for some time, but one I’d never been able to reproduce. Until three days ago.

Analysing an Archi user’s specimen file I discovered that some folders were being stored in XML as <element> tags and not as <folder> tags. This meant that some diagram view objects could not be matched to their corresponding model elements in the model tree. This in turn meant that it was possible to delete the referenced model element thus leading to an orphaned diagram node. It turned out to be a dumb logic bug in the model tree’s drag-drop handler. An edge case, but with serious enough consequences. You can view the bug for yourself in all of its glorious stupidity at Github.

Put simply – if you dragged and dropped a folder onto the same parent folder your file could get screwed.

Fixing this bug has been a cause for major celebration. So much so that, after releasing the new version of Archi, I broke open a bottle of Prosecco and immediately emailed those Archi users who had reported the problem with the good news. Tetelestai!

Now that this bug has finally been squashed and a new build published I hope that all Archi users will download the latest version from the new Archi website.