Categories
development test

Your API sucks : testing

Finally, you’re ready to launch, you’ve integrated the API, you’ve decoded the documentation, you have passing integration tests, and FAT passed against their test server, with their test data. You go live. And everything breaks.

Strawman testing

Sometimes the test data or the API out of sync with live, and you find you’ve been testing the wrong thing. Maybe the test server is a version ahead of the live API (if they’ve ignored the advice not to version HTTP APIs).

More frustrating is when the data is plain wrong. When the returned record has { Firstname = “Lincoln”, Lastname = “Abraham” }, or an invalid postcode (did you know, The final two letters do not use the letters CIKMOV) which means the test site may require a postcode that your validator will reject.

Tests doomed to succeed

Sometimes the test data is correct, but incomplete. Imagine a postcode lookup where test addresses are all SW… which means you can’t test for:

  • Flat x/y addresses
  • BFPO addresses
  • Irish or other EU addresses

Or imagine a payment processor test stub that always returns success – so you can’t test for declined cards, or 3DS cards, or check that you only accept debit cards.

If your API supports it, put it in your test, and make it available to users for their tests. If it’s not tested, it’s not complete.

 

Categories
test

HTC One Mini vs One Mini 2

As a wee aside, and because my last post about an HTC phone was one of my most popular, I wanted to do a quick follow-up to my last post to talk about the phone.

I had an HTC One Mini that stopped responding. It was an ok phone but was often slow and occasionally crashed. When it finally died, within warranty, I sent it back to HTC for repair. I was told I’d get an update within 2 weeks, and on the 14th day after it arrived at the repair centre, I was told they had to send off for a new part.

2 weeks after that, I found out the new part was an HTC One Mini 2, which is one of the stupidest names for a phone I’ve come across. At least the Hero, Desire and Incredible made some kind of aspirational sense.

Having lived with this new phone for a few months now, I do prefer it to the previous phone, but I can’t recommend it.

It’s faster, runs cooler, and lasts longer than the phone it replaces, but the camera is very very bad. That was one thing the previous phone got very right : a responsive camera, that took reasonable pictures. This one is not an UltraPixel camera so the pictures are grainy, the shutter speed is only good for completely stationary objects, and the pictures take far longer to save than they ever did on any previous phone, even when they were saving to cheap sd cards.

More than that however, in fixing the crashes, the software updates have left the phone in a state where it constantly needs rebooted (often via the power+volume-up combo) as the lag time gets intolerable.

I’ve now had 4 HTC phones, and this one will definitely be my last. I’m not sure where to go next though. Maybe LG, maybe Motorola, possibly Sony.

Any recommendations?

Categories
code development programming test Uncategorized

Apprentice programmer

unlit olympic flame

Let’s talk about plumbing, and customer service, and an experience I had recently with a company I won’t name. Overall the experience was good, but I don’t want anyone to get into trouble because they’ve misinterpreted what I have said here. (and if you’re wondering about the picture : the Olympic flame isn’t lit either).

A couple of months ago, our gas combination boiler failed. It refused to start and complained that the flue was blocked. I sighed because we’d had the bearings on the fan replaced last year. So I called the service company to get it fixed.

A couple of days later, I was working from home to let the service technician in. He was fairly young but identified a problem. Unfortunately, a new part was required, so he had to come back the next day. He came back, replaced the part, fired up the boiler and all looked good. The next morning, no hot water and a flue warning 😦 it was the weekend by this point so we got a senior employee, who identified an error in the replacement part (fitted the wrong way) and identified the original problem as a blocked inlet. 5 minutes to identify, 1 hour to dismantle the boiler to get to the blockage and put it back together.

He finished, switched the boiler off, and then made sure it fired when put back together. Problem solved.

So, what did I learn, and why am I posting this on a programming blog?

Firstly, a monthly fee means that I didn’t have to worry about callout or parts, which was great, and left me less annoyed than I might have been.

Secondly, your first fix might not be the right fix. It looked right and it passed the first test, but caused problems later on (at the next restart). Ever written the wrong test and thought the problem was solved? Threading has caught me out in that way before.

Thirdly, just because it passes the test, doesn’t mean the problem is fixed. The boiler fired easily when the case was open and there was more airflow, but didn’t fire when the case was closed. Ever had a bug that only appeared in production or in a system test?

Fourthly, experts will find problems quicker than rookies, but rookies still need to learn.

Fifth, sometimes the only way to fix a problem, even a small one, is to tear everything apart, blow on it, and put it back together again.

The second guy also said something that struck a chord, as I’ve been trying to do it myself : he was disappointed that the company didn’t send the junior out with him so he could pass on his experience so that he’d learn for next time. Apparently, the fault itself is rare, so it’s a good learning opportunity. For developers : fix your own bugs, as there are a lot of things apprentice developers will see in the first few years after they graduate that are rare enough not to come up in university but common enough to be real problems in the real world. For example,
trusting user input (and the related buffer overrun bugs).

So, what have you learned from other professionals?

Categories
code programming test

DDDS2 : TDD? I Don’t Have Time round-up

As a final sign-off on my talk, before it’s released fully to the world, I wanted to discuss the feedback I got and direct people to other thoughts on TDD. It’s all in the interests of openness, so feel free to add any further comments.

First of all, I would like to say that my slide decks and code are now available on the Developer Day Scotland website (Thanks Colin):

Speaker: Craig Nicol
Title: TDD – I don’t have time

2009 Slide Decks – Developer Day Scotland

Feedback

Overall the feedback was fairly positive, and it seems that a lot of people got a good sense of TDD from the demo, although I accept that the pacing was a bit wrong and it was bad form to end the session with some failing test. Still there were some good questions in the Q&A so I’d rather have finished a couple of minutes earlier and had more questions than push on with the coding. I few people suggested that fewer slides and more code would have been good. I’m all for that, I’m a nervous speaker, and it’s been a while since I last presented, so more typing and less speaking works for me, especially when it gets interactive.

I do accept that I probably spoke too fast, and it’s not the only time I’ve been told that, but I hope that will resolve itself with a bit of practice and some better breathing.

For the person who wanted a better comparison of tools, I wish I’d had the time for that, but all the tools worth using have free trials, so have a look at this list from my previous post and try them out yourself:

I don’t know what tools to use

Test Driven Design : Developer Day Scotland 2 « Craig Nicol’s Weblog

For the person who wanted a take home message to answer the point over “I don’t have time to test”, I think I mis-sold the talk a little, and I should have made the mythbusting part more prominent than my original catchy title. I hope I answered a few points about why it can save you time in the long run, but my apologies if I did mis-lead you.

Thanks to Richard for this review. Glad you found the live coding helpful.

10:45“TDD? I don’t have time” by Craig Nicol Craig talked about TDD from baby steps and I gained a lot from it. Was a real shame that the tests he wrote ultimately didn’t pass but the promise of code to finish off gives me something to dig into!

irtimaled: My #dddscot Sessions

Other Resources

If the talk has whetted your appetite for TDD, have a look at the books linked from my previous post and have a look at these three links to help you join the discussion.

TDD Tests are not Unit Tests
Let’s start with the distinction between TDD tests and unit tests. On the surface, TDD tests are very similar to unit tests. This is not surprising, since you use a unit testing framework such as Visual Studio Tests or NUnit to create both types of tests.

TDD Tests are not Unit Tests

good design is far more beneficial and important than the practice of unit testing. However, good design + TDD/Behavioral Driven Development/Acceptance Test Driven Development? That’s where the real rewards happen. Don’t settle for mediocre efforts. Do both and reap the rewards.

Design and Testability – Jeremy D. Miller — The Shade Tree Developer – CodeBetter.Com – Stuff you need to Code Better!

If you do want to dive into TDD, you should also look at Behaviour Driven Design (BDD). I’ve heard it called TDD without the T-word, or an extension and refinement of the philosophy, but however you see it, if TDD is your thing, you owe it to yourself to have a look at the BDD road to design and testability. This link has a lot of good onward links, and it’s a podcast so you can listen while you work, or on the bus. It’s a bit more interesting than Eurovision.

This week on Herding Code, Scott Bellware educates and entertain as only he can. Scott talks about Behavior Driven Development (BDD), Test Driven Development (TDD) and Lean Software Development, gets “all preachy” and donates to the show a nearly endless batch of outtakes.

Episode 42: Scott Bellware on BDD and Lean Development | Herding Code

Round-Up

Thanks again to all the organizers and attendees of Developer Day Scotland.I’ll shut up about TDD for a bit, unless anyone’s got any questions. If you decide you want to use this presentation, it’s open via the Creative Commons license, but please let me know, as it’s interesting to see where it goes.

Blogged with the Flock Browser

Tags: , , , , , ,

Categories
code programming test

DDD Scotland roundup : TDD talk available for download

Thanks to everyone who attended my TDD talk at the weekend. I’ve already got some good feedback, and I accept that maybe I didn’t have a clear enough title or take home message. It was about more than just not having enough time, but the take home message of TDD should be that the overall cost of development will depend much less on debugger and tool performance and much more on the quality of your tests, which is something you have far more control over.

The final cut of my presentation and code is available via the ScotAlt.Net code repository, and it will be available from the ddds site. It is also current available at: tdd-i-dont-have-time.zip

I’m still following the #dddscot and #tdd streams on twitter if anyone wants to send feedback via twitter, or post a comment here.

If anyone can make it to Glasgow city centre this week to talk development, you’ve very welcome to join us at the ScotAlt.Net meeting in Waxy O’Conner’s in Glasgow this Tuesday 7th May at 7:30.

For anyone at my alt.net or horn grok talk, or who was interested in the links at the end of my talk, here they are again in clickable form :

Amazon referal links for the books referenced in the talk, other book sellers are available:

Blogged with the Flock Browser
Categories
code google programming search test

Developer Day Scotland 2 is tomorrow

OK. Slides are written, code is prepared. Laptop is charged up. Hopefully I’m ready for tomorrow. The final cut of the presentation is quite different from the one linked in my last post. I hope to be able to get it up for sharing on Google Docs, but it is available via the ScotAlt.Net code repository, and it will be available from the ddds site.

I’ll be following the #dddscot and #tdd streams on twitter if anyone wants to send feedback on the day (I may be nervous now, but I’m always willing to learn). I may not be able to respond to the feedback straight away, but It’s always gratefully received.

Hope to see lots of folk there, and be sure to bring your questions. If I can’t answer them, I’m sure someone in the audience can, or bring them along to the ScotAlt.Net meeting in Waxy O’Conner’s in Glasgow this Tuesday 7th May at 7:30.

Blogged with the Flock Browser
Categories
code programming test

Developer Day Scotland : Talk refactoring

An update for anyone following this. I’ve decided to refactor the talk linked to from my last post. I’ve created three sections to try and create a better flow and justification. The first act sells the idea of testing itself. The second act looks at the test-first approach, and the final act looks at Test-Driven Design as a methodology that builds on the the ideas of testing and test-first. I’ll post the updated talk up once I’ve had a chance to batter more shape into it. Hopefully the talk will now build up, and people who only pay attention for the first 10 minutes will hear enough to keep them interested for the next 10, and then the next 10… (Yes, I did enjoy proof by induction thank you)

Blogged with the Flock Browser
Categories
beta code google programming test

DOOR Oasis Office Reporting

I’ve been hinting in my status in various places (twitter, facebook, etc..) that’s I’ve been playing about with an idea to do some database/web service reporting using the ODF format, powered by a bit of XSLT. The grand idea is that ODF editors are easier to use and to get hold of than the existing reporting frameworks, ODF is reasonably easy to edit by hand if things go horribly wrong (or you need to change a server name), and, as an XML format, I can use my data to change anything in the output, content, styling, could even generate different letters for US and EU customers by selecting paper size based on country. It’s still very much in the requirements capture stage at the moment, but if it sounds like something you’d be interested in, hit the link below and have a nose around. There’s a discussion group too, which I might have to tweak if things get popular, so pop on over and give me your thoughts.

door-reports – Google Code

Blogged with the Flock Browser
Categories
Blogroll code richtext test

That’s right, I’m watching you

The image shown here is a PNG export of an original SVG icon I created in Inkscape. I’m still looking at the options for this at the moment, but I know that Gecko browsers support SVG properly (at least when loaded from file:/// links), but GMail messes up the link somehow so the same graphics are treated as bitmaps when sent via GMail, which isn’t very handy. I’m not aware of any blog hosting service that will host SVGs (although someone out there may be able to enlighten me), so I may be stuck with using PNGs for the moment, either at high-resolution and scaled down for smaller cases, or creating a bunch of pictures at different resolutions, which does sound very 21st century to me. Ah well.

Technorati tags: , , , , , , ,

Blogged with Flock

Tags: , , , , , , ,

Categories
beta blogger test

Test Post

Using external posting tools to post to my Blogger blog.

C.