Earlier this year, I was thrilled to meet Teresa Koberstein, an arts administration consultant putting Designing Quality Survey Questions to use in her practice. Teresa reached out to learn more about the text and how it came to be, and we quickly realized we had at least a couple of research-minded mutual acquaintances in the local arts administration and education communities. We had a great conversation, made plans to stay connected, and I asked Teresa if she would let me share her story via an interview-style blog post. Here is what she shared!

Can you tell me more about you and your practice? 

I founded Assembly Arts in 2019 primarily to offer feasibility studies and business plans for arts facility development. I work with partners such as Victor Gotesman Performing Arts Planning on projects like the development of the Denver Performing Arts Complex, and have also worked on research projects for Lord Cultural Resources and Fractured Atlas. My academic background is in nonprofit management, theatre and film, and creative placemaking. I started working in arts administration specifically through the National Coalition Against Censorship and Theatre Communications Group. I spent a decade working in finance in the design sector and led west-coast expansion of a large NYC-based public relations agency. I have also served on several nonprofit boards. I try to blend all of these aspects of my knowledge and experience when working with clients today – often creatively approaching their biggest business planning or community development questions..

How did you come across Designing Quality Survey Questions? 

An American Evaluation Association newsletter was advertising upcoming workshops, one of which was your workshop on survey design. It piqued my interest but unfortunately the slots were sold out. Fortunately, I noticed that your book, Designing Quality Survey Questions, was listed on the workshop page and immediately went to buy a copy.  

How do you typically use surveys in your work?

My typical clients range from small arts organizations wanting to take a deep look at their finances to cities across the country that want to develop a new performing arts center. Most projects involve considerable community engagement. Surveys are a tool I use to learn how a new venue could serve the creative community and to quantify their potential usage and technical needs of the space. I also try to learn how meaningful access to a venue might be for the artists surveyed. Additionally, I typically send surveys to the wider community in order to understand audience preferences as a means to shape the new venue programming and ensure it is responsive to those community needs.  

What survey challenge(s) were you experiencing when you found Designing Quality Survey Questions?

I was working with a film and media client on development for a new space. Through engaging via one-on-one interviews with over 70 stakeholders I gleaned incredible information about what this new space would mean for them and some examples of the problems it would solve. There were 8 common themes across these interviews that together made a compelling case in favor of the project. However, when I developed the survey instrument (which I used to reach a wider group of film and media creators beyond the interviews), rather than employing these 8 themes as potential responses to the question, “How or why are new or improved facilities important to you and/or your community?” I used a template from previous venue development surveys.  

This was a huge miss, because although those template responses are applicable to most development projects, this was not a typical development project. In this instance, survey respondents basically selected “all the above” on generic reasons instead of helping me zero in on what this development would truly mean for them more specifically. I wish I had been able to capture how the wider film and media community felt about those 8 themes beyond what was shared with me via interviews.   

I love that example – we’ve all been there. How did Designing Quality Survey Questions help address that challenge?

One of the first lessons I learned from the book is how themes that emerge from interviews can be used to create response options! It was at this point that I realized why the responses from that question in my original survey didn’t yield the answers that I had hoped. I really appreciated how the book pointed to the practice of empathy as one way in which we can develop survey questions and more pertinent response options. I realized that if I had empathized with what the respondent was thinking when filling out the survey, I would have remembered how those 8 themes emerged from the interviews. 

Are there other ways that Designing Quality Survey Questions has informed your work?

Absolutely! For my very next client, I got to test whether or not the guidance in the book could improve my survey development. I was conducting a strategic plan for an arts center in New York. They wanted to learn what differences, if any, their local vs regional audiences experience at the Center. I leaned into the survey design process outlined in the book for creating the following: overall survey purpose, big picture questions, and a schedule. The purpose and big picture question were useful in helping the client feel secure that I, as the consultant, had understood their pressing questions and needs for the survey. The schedule helped us all get on the same page about timing expectations. And finally, I made sure to craft survey questions after I identified themes from the stakeholder interviews in order to shape any choice responses – this was particularly useful when asking for folks to share reasons why they do or do not attend events at the Center. The results were a much clearer picture of audience perspectives and yielded answers to those big picture questions from the client.

Awesome! Is there any particular piece of advice that is sticking with you from the book? Something you want to remember or share with others, perhaps.

The point about utilizing empathy in order to craft survey questions and responses was a huge lightbulb moment for me. I also loved learning about how to measure levels of satisfaction (very unsatisfied to very satisfied) and have since used this method both with a neutral middle point and without it. For the purposes of measuring levels of satisfaction, I think going forward I would stick with the middle neutral point in order to compare responses on a 1 to 5 star rating scale since that is more common across your typical customer review, rather than 1 to 4 stars. 

Thank you so much to Teresa for sharing these insights! Examples like these are really helpful to Sheila and I as we think about how to support folks in applying the advice in Designing Quality Survey Questions. If you have a similar (or really different) example, please leave a comment or reach out to us via email (kfirthleonard@gmail.com or sbr@sheilabrobinson.com) to share more.