Some of the points in this article are correct (e.g. UCD isn't applicable to every single design decision), but the main thesis depends on a couple of false assumptions.
He asks, "Which of the following ways would you think gave you the most valuable feedback?" He submits that almost everyone would agree the real hammer will provide more valuable feedback than the paper or foam ones. This part is true. The next statement is false. He states that UCD practitioners would have chosen differently had the product been a web site instead of a hammer. They would have chosen the paper prototype, he says. That's just not true. UCD advocates the use of paper prototypes, not because they provide more valuable feedback, but because they are much easier to build. This means that you can test more of them in a shorter period of time. That's a very important distinction.
His graph of the UCD process essentially depicts it as a waterfall process, where there are several design steps with big Implement and Deploy steps at the end. That's also usually far from the truth. UCD is very iterative and, if done correctly, involves launching early and using data from real-world usage at each iteration.
It's easy to criticize something if you fabricate its weaknesses. The title of this post is pretty ironic.
I expected a thought-provoking read, but instead I get the impression that the author doesn't really understand prototyping, or the design process in general. Maybe the end has some brilliant revelation that makes it all worthwhile - I wouldn't know because I stopped reading about a third of the way through.
A couple specific gripes:
* That hammer analogy is just awful. A paper representation of a hammer loses all functionality of a hammer. A paper representation of a website loses very little. Also, which would be worse - to draw a hammer for someone and have them say "why does the claw not have a space in the middle for a nail?" or to have them say that after you've spent a month creating your prototype?
* "According to studies by J.Nielsen, users don’t scroll... None-the-less, users scroll more than ever..." ...and so clearly Nielsen doesn't know anything. Except he has long since stated that his original scrolling findings are not nearly as true any more. Times changed and users started scrolling. Anyone repeating that trope is just out of date. Anyone like, say, this post.
The author presents a strawman version of UCD and then tears it down but there is one important point in the article: UCD really hasn't fully come to grips with the new era of web applications.
Everything taught in schools about UCD leads up to the final prototyping stage but there's very little discussion about how UCD applies after a product is launched. A large part of this has to do with much of UCD research and literature being driven by academia.
There's a whole host of amazing UCD stuff you can do with a live web app: A/B testing, polling, behavior tracking, logging etc. However, academics typically don't have access to applications being used by real users & companies are focused more on being effective than documenting their practices so there's a disconnect.
Some of the most interesting research coming out of UCD right now comes from PhD students in HCI doing summer internships at places like facebook, Google & Yahoo. They have access to interesting sources of data yet are concerned more for scientific validity than effectiveness so they're asking different questions about it.
Where is the strawman? The wikipedia version? Or the way I say it's applied?
The very thing that you point out as being relevant is my point exactly. That UCD should test on customers not users in a real environment.
I am not against any of the mentioned A/B testing, polling, behavior tracking, logging etc. those are the very things I would wish more would do.
The problem is that I am of course stepping on a lot of peoples toes that do exactly what I am after. And there are more than you would think. That is least what my 14 years of working with hundreds of clients have shown me.
I found the initial example to be invalid, especially having worked in "physical" industries (auto parts manufacturing, to be precise). The four choices are presented as if they are mutually exclusive - they are not - and are in fact quite often used together.
Options 1 through 3 are incredibly useful for early prototyping in order to whittle down a large set of potential solutions. Certainly every design process I've ever been a part of has also gone through option 4. To present these choices as if people are blindly walking down paths 1-3 and only them, is IMHO the straw man.
tl;dr version: The article brings up the classic "look what dumb people are doing!" example, but yet I don't see people doing these dumb things that are supposedly being done.
So basically you think hammer manufacturers should do all the design and testing in the dark, then finalise drawings, design and produce production processes and machinery and tool up a factory, forge hammers ... and then show them to the buyer for the first time.
Buyer: "we wanted ball-pein hammers not claw hammers perhaps you could show us a product drawing next time"
You are confusing what kind of product the customers want with testing a product you have decided to try and make.
In so far as producing the first hammer yes they should do it in dark. That have nothing to do with whether they should research if there is even market for it. But it is not done by asking the users to look at a paper prototype of the hammer.
The straw man is the "typical UCD process" diagram you put up. UCD doesn't just lump all of implementation into a black box like you imply.
There's nothing in the UCD process that precludes testing on real customers. But the point of using UCD before testing on real customers is that it's a far cheaper way to provide basic realignment.
Can I ask if you've ever done a paper prototyping exercise before? I've lead paper prototyping sessions both in classroom & commercial settings and it's common to find people who question their value before it's performed but rare to find anyone after they've experienced it. Often, paper prototyping sessions lead to a drastic reworking of the interface because the originally designed interface is practically unusable. It's far cheaper to get this feedback on paper than on final implementation, even as the cost of implementation continues to fall.
I have done paper prototyping and any other possible permutation you can imagine of prototypes including developing fully fledged prototypes.
Again what metrics are you setting your "drastic reworking" up against?
How do you know that it needs reworking just because you have found issues in a paper-prototype? Issues that might be addressed in the visual phase or in the final phase.
That is if anything the strawman IMHO. You are solving the very problems that is created by it being a paper prototype instead of the final product.
If the product is a website and the client decides at paper prototype stage they want 3 colums not 2, or they want a right-hand menu or a top-hand (!) search box then no amount of polish is suddenly going to add those things if they client doesn't tell you so they can be put in the design.
Also producing fully finished designs for choosing between - aren't you completely ignoring economic reality there?
While I agree with lots of what's being said in this, the initial analogy really leaves something to be desired. Obviously, the concept for the very first hammer went through a design process as what's been charted out in the blog. Man needs to pound one thing into another to hold it in place. Man tries rock. Man asks around to see what might work better. Etc. The process just didn't linger at the initial steps as long as the final step because the physical application of the tool is such an important thing to demonstrate.
In digital, where much of the initial design surrounds process and more abstract concepts, we can hang out around the early steps longer and to greater effect.
He asks, "Which of the following ways would you think gave you the most valuable feedback?" He submits that almost everyone would agree the real hammer will provide more valuable feedback than the paper or foam ones. This part is true. The next statement is false. He states that UCD practitioners would have chosen differently had the product been a web site instead of a hammer. They would have chosen the paper prototype, he says. That's just not true. UCD advocates the use of paper prototypes, not because they provide more valuable feedback, but because they are much easier to build. This means that you can test more of them in a shorter period of time. That's a very important distinction.
His graph of the UCD process essentially depicts it as a waterfall process, where there are several design steps with big Implement and Deploy steps at the end. That's also usually far from the truth. UCD is very iterative and, if done correctly, involves launching early and using data from real-world usage at each iteration.
It's easy to criticize something if you fabricate its weaknesses. The title of this post is pretty ironic.