Skip to main contentOctopus Think

WordPress site building study

An exploratory user research study, conducted in the open and with community participation, to uncover users' mental models relating to building websites.

Project overview

Problem

Despite its ubiqitousness, the WordPress team knew little about their users. Product decisions were made based on instinct and feedback from the community, most of whom are far more technical than the typical user.

Solution

Sarah designed and lead an exploratory research study to learn more about WordPress users and their mental models of building websites.

Outcomes

We trained and mentored a team of 25 researchers in order to uncover and present actionable insights. These insights are now being used to inform and drive product decisions and approach problems from a user perspective.

Our contributions

user research, user journey mapping, affinity mapping, leadership, community management, documentation, data visualisation

Move to next panel
A visualisation showing the steps in the user journey, including emotional highlights.

Metadata

Introduction

The challenge

WordPress is the world’s most popular content management system, powering over 33% of websites. Unfortunately, for much of the project’s history, many product decisions were made without any real contact with end users. Product decisions were been driven by the needs of the WordPress core community, who represent a very small (and rather biased) slice of WordPress’ userbase.

Move to next panel
Gutenberg logo and list of reasons to do user research: To create designs that are
truly relevant to your users. To create designs that are easy and pleasurable to use. To understand the return on investment of your user experience design.

Metadata

Making a plan (or two)

Project management

After announcing plans External link to the open-source community, Sarah led a small team to collaborate on a project plan External link.

This clarified the project goals, timeline, methodology, target participants, and deliverables, as well as ensuring that our understanding of the requirements was correct and everyone was on the same page.

Move to next panel
Excerpts from the research plan document (linked in accompanying text).

Metadata

Keeping safe in open source

Privacy

The nature of open-source demands that work be transparent and open to outside contributions. This was a bit of a challenge for user research, where researchers invariably have access to private user information.

Working with a lawyer, we crafted legally-binding agreements for both participants and researchers, ensuring we had a legal framework to protect everyone’s privacy.

Move to next panel
WordPress researcher legal agreement

Metadata

Making a list, checking it twice

Preparation

We facilitated an open session to learn questions that designers and developers working on the project had regarding the site building process. These questions then formed the backbone for the interview script. We worked with a recruiter to find participants, and supported community members to join as researchers, even if they didn’t have prior experience.

Move to next panel
Post on make.wordpress.org inviting community to participate as researchers

Metadata

Making new friends

User interviews

Since our researchers were new to the research process, we created guides for each individual role and mentored new researchers in their roles.

All sessions were run remotely following a loose interview script External link. Sessions consisted of a preliminary interview, followed by an experimental excercise, followed by more follow-up conversations.

Move to next panel
User interview script and a screen grab from a session recording

Metadata

Trying to make sense of it all

Data synthesis

Once we’d finished talking to 17 users, we had a whole lot of data to sift through. Sarah lead the more community researchers in the analysis, encoding qualitative observations in a more quantitative way. We reviewed the data in different views to determine patterns and behaviour-based segmentation groups.

Move to next panel
Screenshots of a database of research insights

Metadata

Making pretty graphs

Presentation

Sarah published a report of the results External link as a five-part series, giving an overview of each defined segment as well as key findings, an analysis of recurring themes, and product recommendations. This was followed by an interactive walkthrough and Q&A session External link.

Move to next panel
Screenshots from user research report, highlighting user journey map.

Metadata

What did we accomplish?

Outcomes

Results from the study are being used to inform product decisions across the WordPress project, and to form a foundation for further user research.

Sarah led and mentored a team of 25 researchers, many of whom were new to research. In addition to learning a great deal about real users’ behaviours and expectations, this drove community interest in participating in and conducting research efforts.

Document showing steps in user research project, plus stats (25 researchers, 7 days, 17 participants.)