Skip to content

surveys and survey software

My group is doing a survey for SI682. I’d rather be doing all-interviews, because I like them better and because I actually know a bit about how to do them. Time is really very short right now, so we’re trying to supplement with the survey. I get butterflies in my stomach because, while I have taken many surveys, I don’t know a thing about writing them, but I suppose this is part of how to learn. You can help our group by taking it. I already realized I forgot to ask where people live (oops), so if you want to fill that in on the last question, that’d be swell.

In the process, I searched the Internet (well, mostly SourceForge) high and low for a tool that would do what I wanted. In the end, there were four that I really wanted to try, and only one that survived the test.

The first I tried, and most promising, was Web Survey Toolbox, from CMU’s HCII. It looked quite good, but unfortunately requires Tomcat, which my host does not support. I’ve also had some unpleasant experiences with Tomcat.

From there, I moved on to try UCCASS. It looked simple and straightforward. Install was quick, and I had created most of the initial survey before I realized it didn’t have a way to view individual responses (so there would be no way, for example, to correlate an answer to one question with that user’s answers to other questions). I could have written code to pull that from the database (I think), but I didn’t want to have to write new code to do this.

I next tried PHPSurveyor. This one gets used a lot. Aside from icons that are an eyesore and require you to mouseover them and wait for the ALT text to figure out what the link does, it looked like it had some good things going for it. Install was reasonably straightforward (though not without some .htaccess troubles), but what killed this project was the steps required to take the survey. Users must either have an email token inviting them to take it or sign up online and wait for a token to be emailed to them. While this is probably great for many situations, I didn’t want to make people deal with this, so it was time for the next application.

This happened to be phpESP. It’s pretty simple, and has limited answer types. With some concessions on my part, though, it could work for this survey. It has some serious shortcomings. For example, if you miss a required question, the page will just appear to reload with no message indicating why. This gets frustrating pretty quickly and it can take a while to realize what’s going on. Still, it’ll work for now.

Have you found something better? I’d love to hear about it. There were a number of apps on SourceForge, but these were the ones that seemed to have the most documentation online, and rather than downloading and installing everything, I tried to go with the ones about which I knew the most.

{ 5 } Comments

  1. Noor | September 24, 2006 at 9:06 am | Permalink

    Most people at SI use SurveyMonkey – it is pretty cheap and does a lot of analysis for you. UM also has some sort of a form script but I can’t remember what it is called.

    You’ll learn more about surveys in 622 but here is some constructive criticism:

    1. You should checkout Observing the User Experience by Mike Kuniavsky. He has a lot of information about surveys and includes some sample survey scales.

    2. The scale on question 9 has some problems. “As much as possible” is a different measure than “frequently” (intent vs actual behavior). For instance, I want to walk to places as much as possible but I do it rarely.

    3. Open ended questions (pages 5,6) would be better as interview questions. Open ended questions are really hard to analyze in a survey and they place too much burden on the survey taker. I almost stopped taking your survey when I saw them.

    4. For this kind of design research, I think you might be better off doing a few interviews (even if they’re just with friends) than a survey. A survey is good for getting statistically significant data about a large number of people. However, at this stage of your research, I’m guessing you want to learn more about your users’ pains and interviews are going to give you a certain depth that surveys won’t.

    5. You should pilot your survey with other people who are not in your group – do they understand it? You might want to instill the help of some second years who have already taken 622.

    good luck!

    (is this for the CHI 2007 SDC?)

  2. sean | September 24, 2006 at 1:02 pm | Permalink


    Thanks for taking the survey! Couple of quick thoughts and questions.

    1) After reading a couple of reviews, I added the Kuniavsky book to my reading list. The recommendation is much appreciated.

    2) You’re right.

    3) Yes — I think this was a manifestation of my “I wish I had time for more interviews” feelings, and I had concerns about putting them in. The survey ended up largely as a modified version of our interview protocol with some other questions jammed in. Deletion of some of the open-ended questions probably would have been wiser.

    4) I agree, at least in principle :-). We’ve been doing interviews alongside, but — after looking at the results as of Weds — felt a need to cast a slightly wider net. It’ll be interesting to see if, in the end, we really feel that the survey added to our knowledge or if it was largely a time sink.

    5) Yeah, I did a quick run-through with a friend from home, which fixed some major problems. Getting some input from people who have been through 622 is a great idea and will probably use if we do another survey.

    What did you think of 682? It’s feeling very similar to a course I took previously (and I learned just the other day that the syllabi of the two courses are actually based (at least in part) on the same coures, so that explains some things. It feel like a helpful experience, but largely too rushed to get the depth I feel I need to get out of the masters program.

    Yes — this is for CHI 2007 SDC. We’re not 100% committed to submitting it (I think we’re waiting to make sure we all feel good about the product), but that’s the goal.

    CHI-opportunity aside, I think my team also gravitated towards the idea of looking at a broader problem rather than trying to redo an existing tool. We’re finding, though, that the schedule seems to lend itself much more towards the other sort of problem.


  3. Noor | September 24, 2006 at 9:52 pm | Permalink

    622 is a great class . . . perhaps the best hci class at SI. It is rushed, though.

    I hope you do submit your project to CHI. It would be cool if a few SI teams made it to the conference round.

  4. Noor | September 24, 2006 at 10:02 pm | Permalink

    Oh oops, you asked what I thought of 682 not 622. 682 was taught by a different instructor when I took it. I learned a lot from the class but didn’t necessarily enjoy it at the time. Mick has really made an effort to improve it but I can imagine that there will be a few kinks this semester (being that it is the first time that he’s teaching it). It was very rushed when I took it and despite having an awesome team of very talented people who worked super super hard . . . we still did a lot of satisficing. The class is really about interaction design and not visual design. A lot of people expect to learn visual design so they’re disappointed when they don’t.

    Have you heard of Rank SI yet?

  5. sean | September 24, 2006 at 10:24 pm | Permalink

    I’d heard of Rank SI (Lada had mentioned it), but I hadn’t found it yet. Thanks for the link.

    I was poking around Amazon tonight and noticed that Kuniavsky had made its way into my recommendations list (am I that much of an HCI dork? perhaps…), so I decided that was a sign that I should just buy it.

Post a Comment

Your email is never published nor shared. Required fields are marked *