Poorly surveyed

 

I got this email just now, it’s so ugly I thought I’d follow the link to see how bad the survey was. I took the extra time to take pictures and write comments.

The landing page as not too bad, except I couldn’t care less about adding to my nano collection, and suspect many in the target market feel the same way.

The first questions were broad. So I own lots of devices – but which ones do I view news on? The categories of apps are so broad. Perhaps it wold be better to focus on media apps, and even ask which ones I have and which ones I use. (NYTimes.com is the only one I use, and even then not very often)

The questions stayed bad – it’s actually really hard to design a questionnaire, and I suspect that the answers from this one will not be that useful. The feature list forgets that the most interesting thing is content. The payment question could lead to someone thinking that $10 per month is ok, when we all expect to pay $1 for the app and that’s it. And even the publications question – the answer it I’d probably look at all of them if they were in one app, but more important is the link to their online content.

I like the open question about what a mobile app would look like, though answers could be quite fun.

Finally the survey makes the basic mistake of enforcing the question asking for my email address and phone number. This is a great way to reduce response rates and increase bias, as evidenced by my not hitting the respond button.

So to summarise:

– We want to help, and while some people are motivated by winning prizes, most are not. So don’t make us give you our email address and phone – it feels like spam and we probably won’t.

– Focus the survey on the topic – which is a iPhone/Android/iPad app for your magazines

– Write the report that you want to result from the survey before you write the survey. This means that the answers you get will be the answers you need to fill the holes in your report.