Dear all,

I have now completed the full survey report and am attaching it to this post (in pdf and Word format).
It starts with 2 pages of summary and then provides all the questions and answers with analysis/interpretation.

I copy here the general conclusions from the report (based on the 435 survey responses that were received):

General conclusions:

1. Overall, the survey shows that the users are quite happy with the Forum. There seems to be no “loud calls” for fundamental changes.
2. Like with any public online forum, the Forum users are made up mainly of people who read and follow discussions and to a lesser degree of people who are active writers. So any improvements made to the Forum should be with this in mind attempting to increase the number of active contributors as it is these contributions that make the Forum lively and interesting.
3. Improvements could be made regarding a new front door (specifically aiming at newcomers), an improved menu bar at the top of the Forum page, easier to use search field, easier to access Google Translate button, re-arrangement of the Forum categories to bring out the less technical topics better (e.g. political processes, behavior change, health-related topics, sanitation marketing and business aspects, scaling-up, global political processes).
4. The following suggested new features are all potentially popular (in decreasing order of popularity according to the survey): summaries of long threads, periodical structured discussions with experts on the Forum, live virtual events (webinars, Google Hangouts or Twitter chats to supplement Forum discussions), rating posts, and a new question-answer category.
5. The high-quality work of the Forum moderators is very much appreciated by many users and is understood to be critical for this kind of forum.

Some further conclusions from the report that you may find interesting:

User characteristics and preferences

• There is a good spread of professional contexts represented amongst the Forum users. Most of the users are working for NGOs (35%), followed by academia (17%) and the private sector (16%). The last third are students, people working for the government, interested members of the public and “others”.
• “Learning from people in the sector” was by far the most quoted reason for using the Forum, followed by “professional networking” and “getting help and information for specific questions”.
• The e-mail digest is the main place where people read new posts, and the vast majority (67%) of survey participants found the Forum e-mail digest to be highly valuable.
• Six percent of the respondents read posts on a daily basis and seven percent post weekly or more often. However, a “typical” Forum user, according to this survey, is more likely to read posts about once a week and to write posts less often than 4 times per year.
• In questioning how well the Forum meets certain characteristics, all characteristics in the answer choices received high scores. The differences between the six characteristics were only slight, but the three highest-scoring characteristics were “interesting topics”, “friendly, supportive atmosphere” and “well moderated”. Slightly less approval was given for the three characteristics (in decreasing order): “high quality of posts”, “easy and intuitive to use” and “good networking options ”. Therefore, these are the three areas where the Forum moderators and organisers could think about possible improvements.

Forum categories (there are currently seven categories on the Forum):

• Getting the categories “right” is not easy but is important for two reasons: The e-mail digest is grouped by category; and users can subscribe to certain categories to receive new posts in that category by e-mail.
• The category “Sanitation systems” was clearly the most important category for the survey participants. In second and third place were ranked the categories “Hygiene, health and schools” and “Science and technology”.
• Based on feedback by the users the category “Sanitation science and technology” could be combined with the “Sanitation systems” category.
• The sub-category on governance (and related topics) should be distinguished from the miscellaneous category. It might be better to rather combine the category “Miscellaneous” with the announcement category.
• The category “Hygiene, health, schools and training/education” will need to be re-assessed as it combines many topics in one category.

New landing page, black menu bar, search field:

• Giving the Forum a new landing page was supported by about one-third of the respondents, and many ideas were put forward. The two most popular new features for a new front door page are (several answers were possible): “Links specifically to assist newcomers” and “Forum’s hot topics of the week” which were each selected by over one-third of the participants. On the other hand, the interest in “Selected (or curated) reading lists by category” and in “Top contributors” was relatively low at 19 and 17% of respondents.
• In connection with the new landing page, it was observed that the black menu bar on top of the Forum page is not working as well as it could – only half of the respondents said they found the black menu bar useful or very useful. Improvements need to be made to ensure people know it is there and that the items are clickable.
• Many people asked for a better search field: directly having a text box with magnifying glass symbol next to it would probably be best.

Possible new Forum features:

• The proposed new features of “Summaries of threads” and “More structured discussions” were the most popular of all the proposed improvements, with 80% and 74% of users indicating that they would find these useful or very useful. It might be possible to encourage Forum users themselves to compile summaries of longer threads, e.g. of those with more than 25 replies.
• The proposals of live virtual events, rating posts on a 5-star system and a new question-answer category were all reasonably popular suggestions, with 58%, 57% and 54% of respondents, respectively, saying they would find them useful (several answers were possible for this question).

Moderation of the Forum:

• A few users complained about “aggressive” language used by other users on the Forum sometimes, e.g. “shooting down” of new/unconventional ideas, not being kind to new people. It was suggested that the moderators should intervene more in such cases.
• A lot of praise and thank-you notes were received for the moderation work. Many users pointed out that good moderation is key to the success of such a forum. Users appreciated, for example, the “sensitive yet never bland moderating”; “knowledge is excellent, perspective is good, answers are direct, insightful and concise, good insight into what is being discussed where [in which location] on the forum (can refer to other posts/move threads/elaborate on topics nicely)”; “moderators appear very informed and keep discussion flowing”; “open positive attitude”.*

Once more, thanks for your participation in the survey!

Feedback on these survey conclusions is welcome, as always.

And stay tuned to see upcoming improvements based on the results of this survey!

Regards,
Elisabeth


* One person asked "How many forum moderators are there?". As you have probably noticed I am currently the main moderator of this forum but there is quite a team supporting my efforts (amongst others Shobana Srinivasan, the current intern at the SuSanA secretariat), see this forum post from last year here : forum.susana.org/forum/categories/135-qu...orum-being-moderated . And we also have a Forum Committee consisting of the Featured Users and the moderators. This Forum Committee is called upon when there are tricky issues to be resolved.

P.S. The new front door which was mentioned in the survey is now available, see here:
forum.susana.org/forum/categories/10-ann...nd-new-layout-inside

Dear Florian,

Thanks for your feedback! And yes, it was a busy month of October for me, Shobana and Arno with the survey advertising on all channels, e-mails, encouragement and prodding to get to the 435 responses in the end. I am proud of this achievement.

Your post shows how the same data can be interpreted in different ways : For Question 23 about the Featured Users my conclusion would have been that 44% of people liked it or liked it a lot! And that only 19% didn't like it. (and 37% of people are indifferent about it)
A few of the comments for that question said they really liked the interview part about it...
(I am still going to post the comments later)

And about the "game-like gimmicks", it is true that the "rating users" did not come out high in this survey. Perhaps people also don't like the idea of being rated themselves. (the "rating posts" option on the other hand was more popular).

But we have no way of knowing whether the huge number of people who read quite often but who don't post often could be encouraged to post more often if there were more "special effects" from the gamification repertoir. This is one question we didn't ask: "What could be done to make you post more often?" The answer I hear most often from experienced people is "I don't have time" or "I don't want to make time for this" or "If I need to know something, I just ask some of my colleagues in my network by e-mail.", and the answer from inexperienced people is often "I have nothing yet to contribute / I am intimdated by the high level of expertise by other people / I don't want to say anything stupid that I will be embarrased about in years to come".

I think though that a better front door page and some other small improvements (tweaks, as you call them) might lower the hurdle for posting, especially perhaps for the newcomers.

Any more feedback by anyone?

Cheers,
Elisabeth

P.S. Mughal, I have already posted my report on Part 1 of the survey on 7 Nov (scrol up); and the written report for Part 2 I am still busy with. Will post it soon but would also be keen to hear people's interpretations of the graphs (like Florian did) without being biased by my interpretations of the figures.

Dear Elisabeth,

That is an awful lot of work that you have done! That is great.
You have, however, presented the results. Where are your evaluations and findings?

Regards,
F H Mughal

Thanks a lot for posting the result of this survey! You really did a lot of effort to get people participate and you achieved really a good result with more than 10 %. Well done!

A few reactions or comments to the results:

- Overall, I think the survey shows that people are quite happy with the forum as it is. A few tweaks here and there may be useful, but all in all, there seems to be no big need for improvement
- Q4: 7% of the respondents, that is 30 people, post weekly or more often, the rest once per month or less. Assuming that all active forum users responded to the survey, this means that only about 30 out of 4000 registered members write weekly. This puts the high number of members a bit in perspective. 30 really active members is still not bad though...
- Q13: language seems one part where need for improvement is really felt... and this without asking the people who really have problems with the language.
- Q23/24: more than half don't like or don't care for the "featured forum member" and "rating users" is the only unpopular one of the suggested improvements. Seems people don't want to rate other people or be rated. My interpretation is that people come to Susana really for professional discussions, they don't really want game-like gimmicks.

Ok, looking forward for more results and your analysis.

Best wishes,
Florian

Here comes Part 2 of the survey results. Part 2 was about "Your opinions and ideas for improvements".

I am posting just the graphs on their own first, and my report with observations and comments later. Plenty of comments were also made in the free text fields and I am trying to condense them or just pick out the most important ones. This will come in a subsequent post.

Meanwhile, what do you think of these results? Do you find any of it surprising?


Regards,
Elisabeth

Dear all,

You are probably eagerly awaiting the results of our first large forum user survey which we ran during October. I am happy to bring to you the first part of the results now!

Firstly, let me thank the 435 participants who took the survey- thanks so much!! We have surpassed our aim (which was 10% of all SuSanA members, or 400 participants). All participants who left their e-mail address to take part in the prize lottery should by now have recieved an e-mail by Shobana from the SuSanA secretariat who is kindly sorting out the prizes to match as closely as possible your desired prize with the prizes available! Thanks again to SEI, GIZ, Eawag-Sandec and SHARE for making these 150+ prizes available.

In this report I have compiled the results of Part 1 of the survey (Part 1 was on "Your preferences as a Forum user"). It is so far just the raw results and my first set of observations but without an in-depth discussion of the next steps we should take (this will take a bit more time to develop):




I invite you all to read through my observations and tell me if there are other important observations to be made from the data that I hadn't thought of yet? And please tell me your own conclusions from looking at this data?

I have also copied some of the graphs from the report below, in order to spark your interest in opening the pdf file or - if you don't have time to look at the pdf file - to give you a quick overview of the resuls of Part 1:






















The results from the second part of the survey questions ("Your opinions and ideas for improvements") will come by Monday, as soon as I am through with it all. There were so many comments in the comments fields, it is taking me a while to wade through them all!

I think we have a really rich set of data here and I feel that I got to know "you" and "your preferences" a bit better this way.

Enjoy the reading and I am looking forward to your feedback and observations if you have any?

Regards,
Elisabeth

[Start of Page 2 of the discussion]


Dear Mughal,

Yea, sorry, the Word version of the survey doesn't look very pretty but it still would work (it would take me too long to make the formatting nice, given that only a dozen people or so from China would probably take the survey by using the Word document? They could also use the pdf file: print it and then place their ticks, then take a photo with their smartpone and send me the photos. I know it's awkward but I can't think of a better solution for those who cannot access Google forms).

Surveymonkey is a really nice online tool to do surveys. The basic version of Surveymonkey is for free which is why many people like to use it. The advanced version with extra features costs money (but not very much).
See www.surveymonkey.com

I used to use Surveymonkey for surveys in the past but now that I have discovered Google forms thanks to Roslyn (completely free of charge, no limits on questions or on participant numbers (or if there are limits, the number is very large)), I will stick with that one for future surveys that I might be involved in.

Regards,
Elisabeth

Dear Elisabeth,

Thanks for caring about the Chinese friends. Word format is not quite in order. Please ask your IT guru to improve it. No need to rush in - take your time.

What are those "surveymonkey" surveys? I get those from other reputed organizations.

Cheers,
F H Mughal


[End of Page 1 of the discussion]

Dear all,

We have now reached nearly 400 responses to our Discussion Forum User Survey (thank you!!), but we still want more.

It has been brought to our attention that in China people cannot access the survey because:

I can not do it because the survey pointing to google which is forbidden in China.

I didn't know that. Are other countries also affected in this way? (the survey was created using a thing called "Google Forms")

As we really want to hear everyone's voice, I am attaching here the survey also in pdf and Word format. The Word format is not as pretty but the answers could be selected with colour.

Thanks for taking the extra step to fill this in. You can just e-mail it back to me at:
This email address is being protected from spambots. You need JavaScript enabled to view it.

For everyone else, the link to the survey is still the same:
docs.google.com/forms/d/1ZaO6UIuG68XufQh...oJ_fXWPpQH4/viewform

Thank you to everyone who has already participated! The results are looking very interesting, and I will be sharing them with you next week after the survey has closed this weekend.

Regards,
Elisabeth

I know how hard it is to get suffiecent people participating in an online survey...

So I want to join those above in encouranging everybody who has not yet, to take the 5-10 min and fill out the form. Its done really quickly and will help to have a much more meaningfull result of the survey. It's also a good chance to express wishes for new or better forum features.

Regards, Florian

I have participated in the survey. It is a wonderful experience. I encourage all the users to kindly participate in the survey. While the prizes are attractive (like Doreen, I wish I could get all the prizes - they will significantly to my mini-library!!
I would like users to note that the moderators have taken pains in developing this exciting survey, and we are morally bound, as responsible persons, to respond accordingly.

I can't wait to see the results!!

F H Mughal

Hi, everyone,

I love the SuSanA forum and so does everyone I know who uses it. At PHLUSH we get lots and lots of unsolicited questions on topics ranging from public toilet access issues to microbiology. We cannot possibly even try answer them. So we send people to the Forum, ask them to sign up, and become part of the conversation.

Now here's an opportunity for all of us to make the Forum even better. TAKE THIS SURVEY! docs.google.com/forms/d/1ZaO6UIuG68XufQh...oJ_fXWPpQH4/viewform And if you scroll down below you see the prizes! Books include acclaimed favorites and some brand new titles. Hard copies!

Wouldn't you like to get away from your screen and curl up with a great paper book? But first you have to invest 4 or 5 minutes and TAKE THE SURVEY!

Carol


+++++++++
Note by moderator (EvM): We have by now 212 responses for the survey but we are aiming for a response rate of 10%, meaning 400 responses. So please do take part in the survey, even if you have not used the discussion forum so much in the past!