How to make your users not want to murder you

Editor's Notes

• #2: What would cause my users to be so upset as all that?
• #6: This will be very seat-of-the-pants, intuitve stuff “Beginners mind” Feelings rather than measurements Information about essences rather than details Should feel confident when using; should feel doubtful when appropriate
• #9: Psychologically engineered Simple to work with for safe things Psychological barriers for unsafe ones (four steps with three warnings to erase the phone)
• #10: Let’s do a case study of a good software decision but a bad psychological one One of the most popular and most used modules in CPAN Heavily used, much written about it The ‘go-to’ heavy-lifting module for Web interaction
• #11: A very, very specific disclaimer: None of the following is meant to say anything bad about Mech, or Andy, or anyone else who has worked on Mech It’s great, I use it, it got me a job writing Web automation software This talk is meant to point out how a small failure in psychological engineering can cause a surprising amount of trouble
• #12: Canonical example. Documented this way in hundreds of places. All over the net. Lots of books. Especially in the Perl Cookbook. Very standard, ver Perlish, like open(): try the operation, did it work, continue if so
• #13: As of this release, something happened that breaks these examples.
• #14: Here’s the note in the Changelog (As an aside: this is a minor engineering psychology misstep: non-engineers skip the changelogs and install things first)
• #16: New programmers who are not checking the get() calls are unfortunately the people who aren’t reading this!
• #17: Great software fix. Not as good psychological engineering because the audience this is intended for hasn’t read this and is not going to notice the change. It also doesn’t mention the actual consequence of the change!
• #18: So this is the real meaning.
• #19: If the get fails, you no longer get to check the status; get() dies. That’s what autocheck does, and now it’s on by default. This is code “that was just working yesterday, what happened?”
• #20: Unless you’re using a subclass and in that case it isn’t. Inconsistency leads to confusion and is a “cold pricklies” generator. Code that looks identical at the point of use behaves differently.
• #22: A failure occurred, but what do I do now, if I am new to Perl, or a dabbler? All I need to do is add autocheck=>1 ... but I need to know that I need to do that.
• #23: Users who had working code, sometimes production code, suddenly had broken code. Worse, they only had broken code in an exceptional situation: if the get() worked, almost all the time, they’d never see the problem. The API had changed in a subtle way that was not automatically caught if you upgraded the library. Remember that often one person is responsible for the script, and another for keeping packages up to date. In a situation like this, it’s only possible but likely that this would happen. (It happened a LOT at Yahoo!) But surely this was only a problem till the message got out...
• #24: Another example (from a surprising source) of bad psychological engineering. If you held the iPhone 4 the way you were used to, you’d interfere with the antenna. This was the first response. The second was the free cases program.
• #25: The best psychological engineering would have ensured that any attempt to “hold it wrong” would have quickly informed the user of potential problems - especially if “standard”examples could trigger the problem. Most of the time there was no awareness that there was a problem...
• #26: Surely, though, this isn’t REALLY a problem. Right?
• #27: As of 2012, it still was, especially to new programmers, who for some reason were still not expecting this “helpful” behavior. (And the answer here is also technically correct ... but psychologically incorrect. Actually showing how to turn autocheck off would have yielded “warm fuzzies”.)
• #28: Seems to have tailed off now; I can’t find any more-recent failures, which is good! But this still means that it took 4 *years* to get to this point!
• #29: If you’re thinking solely as a programmer this makes no sense. The WWW::Mechanize Cookbook had this in examples on day 1 of the release Why did it take 4 years? Why did it happen at all? Why didn’t the message get across? Why was it missed and why did it continue to be missed?
• #30: We’ve had many, many releases of the module and my code continued to work fine. Why should this one be any different? Mech is awesome, it just works! (Huge amounts of warm fuzzies, leading to) It’s so great, I’ll use it for everything! Like the recovery script I run when there’s a problem! Or my quick health-check script! Or lots of other things I don’t use too often!
• #31: Contributing to complacency: - previous changes had been more thoroughly pre-tracked and pre-announced (good psychological engineering!) This one was different - autocheck wasn’t optional prior to its introduction, and there was not “you didn’t use autocheck” deprecation period
• #32: This was the major change. Note that it didn’t have any hidden behavior: if you (sensibly) had use strict on and tried to access %headers, you’d die at compile time (handwaving eval’s). Great psychology! We’re going to make a change with potential breakage, so we’ll send the signal via the version number too. Some points: - bumping the version to 1.00 is expectation management. - The change would NOT cause an error at run time, but compile time (some pricklies, but not late-night-oh-crap-why-is-this-broken-it-was-working-last-time pricklies)
• #33: Yes, I should submit a patch. I promise I will after this session.
• #34: Here’s a number of relevant hits from a quick Google of Mech->success (5M hits). Only the Mech examples use autocheck explicitly.
• #37: This generally summarizes how my users felt about it.
• #38: We need to tell thousands of people, likely running many scripts each, to make this fix in every one of those scripts How can we do that in a way that does not irritate them while keeping the message in front of them? We need to send the message through as many channels as possible at once We need to communicate as directly to the affected people as possible
• #39: A meta-message, fairly standard in the Perl community (this is why you see things like version 1.294) For the maintainer, a pure engineering solution, with no code involved. Warm fuzzies from knowing something big happened.
• #40: Tell me what is wrong. Don’t make me guess or have to find that mention in the docs. Very direct messaging: here at this place in this code, this specific thing needs doing. Relatively minor software effort; just a longer message. A little bit of wording needed to convey the right impression. The programmer cared enough to tell me how to fix this, so I can just get it it done in a few seconds. Huge warm fuzzies.
• #41: Alternate: at least tell us that a new default is now in force. Doesn’t stop the error but at least lets us know that it can happen. A very simple check, but more intrusive than the expanded die(). A little less programmer-friendly; those who “know better” and “just want to use the module” will get warnings. Appropriate cold pricklies, with an admixture of “you should know” warm fuzzies.
• #42: A useful mitigation: code doesn’t have to be updated yet; we can do back to the old behavior until the problem can be corrected properly. A long name encourages fixing it properly, and a cleverly chosen one is a reminder at the same time. More social than software, though this adds more maintenance burden - the old and new code have to coexist. Extra-warm fuzzies for being able to get old code to work without changing it.
• #43: Emphasize the change; because this changes away from the original default (check actions for success), make sure that anyone even glancing at the man page sees this. Not quite as effective, but more likely to catch people’s eye when they go to the docs when they get an error. Purely a psychological change, not supported in the software; not as good as it’s not at the point of the error, but has the advantage of possibly forestalling the error happening at all. Warm fuzzies if seen; more likely to be seen if done this way.
• #44: Let’s take a minute to look at why none of these things were done. Again, this is LEARNING FROM HISTORY, not blame.
• #45: Many of the bug reports from WWW::Mechanize were caused by people not calling $mech->success in the pre-autocheck era From the author’s perspective fixing this is very important as it’s a large source of bogus errors
• #46: (Here were looking at erosion caused by rabbits in Australia.) The solution fixed a problem that the author had very well But it created a secondary one by sometimes breaking old code And a tertiary one by not making it dead simple to recover from that
• #47: These are all meant to communicate to prevent a problem and to provide quick recovery in case one happens anyway.
• #48: We need to consider both old code and new code, and work out ways to protect all of our users.
• #49: Announce and engage - both before and after the change - Blog about the change; writing an explanation may help you spot an oversight - send email to your user mailing lists (you should have one that people can subscribe to) - Announce the upcoming change in public areas: Twitter, Perlmonks, etc.
• #50: Communicate! - emphasize the change in the changelog (done, but maybe not as much as needed) - put it up front in the man page (ideally at the very beginning so we don’t have to count on users reading far enough to see it) - put it in the commit; this gives you a chance that someone else following the commits will spot the assumptions and tell you - update errors to make sure that new errors explain why they are there
• #51: If a common idiom is different, warn or die as early as possible. Better to die in new() with a diagnostic spelling out the problem than to only die in exceptional situations without a full description of why.
• #52: There is a workaround, but not a convenient one A variable like this helps emphasize that you haven’t done that you need to do Second, it’s a pain to use, so you want to fix it ASAP
• #53: Make sure that what to do about it is clear Add a section to the docs specifically about what the change is and what is does in all cases If there’s a hack to get around it, document it here (“add autocheck=>0” to all new calls, or subclass WWW::Mechanize) Spell out what needs to change to be compatible with the new function (all new code needs to expect that GET/POST will die instead of setting a failed status)
• #54: I initially posted a meditation about this on Perlmonks. The response was almost unanimous that I was over-coddling my users.
• #56: Another camp focused on the psychological problem, but solely as software developers.
• #57: It’s true, but being right about procedure does not fix the problem. What are the reasons it could happen in the first place?
• #58: You’ve got a deadline, you’re rushed, you’re updating other modules; take a quick look, looks fine, install it. Oops, some time later. You got distracted while reading the install docs and missed it, or it flew by in a giant CPAN install. You’re under pressure to get a new version up because you’re behind and this version fixes bugs that you’re hitting. You’re maybe not that smart at the moment it comes time to make the decision.
• #59: This is not necessarily because you are dumb! I like to shoot for about how smart I think I am at 2AM. If something like this goes bad, it almost always happens after hours and after I’m asleep. I am very not smart when I’ve woken up out of a deep sleep - many people aren’t.
• #60: Flight officer fell asleep for 75 minutes and woke up disoriented and “not feeling well”. An unusual situation occurred, and he reacted improperly because of sleepiness, not inability. Several passengers were injured.
• #61: What we want is enhanced usability for those kind of times. Not hand-holding, but completeness and accuracy. At 2AM I need that little extra help and reminder. It’s not expensive to do, so I do it as a safety issue This engenders trust.
• #62: This is how users should feel about your code. If an error occurs, it’s real, not an “oh by the way I changed this, sorry”.