How Card Sorting Helps Solve Fundamental Design Problems

Audio is a big part of the Blinkist experience. Around 70% of Blinkist users listen to our content and when I joined the team last summer as a new product designer, I learned about an exciting new content format the team was working really hard to deliver early 2020: audiobooks! Our goal at Blinkist is to become the destination for the modern, lifelong learner and by introducing new content like audiobooks, it gives users the opportunity to expand the way they learn across a range of different formats.

The app library was originally built to house the 15-minute packs of key insights from books (which we call Blinks) and to support a variety of different features like highlighting text. When I took over and dived into the insights – with a combination of user research, customer feedback, and business analysis – we identified the following problems with the library:

• Users had difficulty finding or choosing content.
• Users struggled to find titles they’ve already finished.
• We were developing multiple new content types that need to be displayed in the library in 2020 (audiobooks and original shows) and the already constrained library would not have been able to accommodate them easily.
• The library wasn’t designed in a way that is easily extendable towards learning management goals.

We met with the research team and agreed a card sorting activity combined with user interviews that would give us valuable insights.

What is Card Sorting?

Card sorting is particularly useful when you need to clarify the information architecture of a product or website. According to usability.gov, “in a card sorting session, participants organize topics into categories that make sense to them and they may also help you label these groups. To conduct a card sort, you can use actual cards, pieces of paper, or one of several online card-sorting software tools.”

Why Did We Choose Card Sorting?

With the new content format, audiobooks, about to be released in the app, we wanted to make sure that both old and new user needs would be taken into account. So far, the design of the Library was pretty straightforward: a shortlist of titles that could be filtered by user progress, with a range of features accessible under tabs.

However, we were also facing other problems with the existing Library such as:

• Lack of discoverability of the filters as data showed they were barely used (even the users with the biggest libraries)
• Low usage of the features as data showed users barely navigate to other library tabs (favorites and highlights)

As we didn’t want to make any assumptions and decide for users whether to prioritize format over progress-state (i.e. not started, started, finished), we decided to combine two card sorting activities, starting and ending with a user interview to understand their behavior.

A card sorting activity provides a good understanding of the users’ subconscious expectations of how the information should be organized.

With this research we were hoping to achieve the following objectives:

• Understand how users group and perceive content within the library.
• Use the results to define the optimal information hierarchy for the library, considering the new content formats coming in the app as well as long-term needs.
• Understand whether all of the functions that the library currently fulfils need to continue to belong in the library, or could become separate.

I can’t thank my teammates enough for helping me write and iterate on the cards, draft the script, moderate or take notes during the sessions. Card sorting can be overwhelming and it’s very important to define beforehand both the goals and questions we want answered.

These are a few questions we needed to answer in order to redesign the library:

• How do users choose/prioritize which content to consume, and how important is it to have a shortlist/library?
• What are the most useful methods of prioritizing/organizing the library (such as filters/search)?
• Whether/why the users find the library useful, and what they think that benefit is?
• What information users need to help them make a decision (reading/ listening time, number of reads, subtext, category/ies, reviews, etc.)?

We defined the following success statements:

• We have a clear understanding of the goal of the library
• What are the most useful methods of prioritizing/organizing the library (such as filters/search)?
• Whether/why the users find the library useful, and what they think that benefit is?
• What information users need to help them make a decision (reading/ listening time, number of reads, subtext, category/ies, reviews, etc.)?

Open vs closed sorting

Open — In open card sorting, each participant is given a stack of cards.The participants are then asked to group those cards together in any way they want. Then they create labels for the groups that they chose. This method is commonly used for new/existing information architecture (IA) or organizing products on a site.

Closed — In a closed card sorting, the researchers create the labels for their respective groups. Participants are given a stack of cards and are asked to put each card into a group. This method is normally used when adding new content to an existing site or gaining a second round of insights after an open card sorting.

What happened when we tried card sorting?

We decided to have an open card sorting allowing the participants to group the cards together and create a label for each cluster.

This study can also be done remotely but I wouldn’t recommend it if you need to ask follow-up questions and dive into more details. We were focusing on the information architecture of the library and we didn’t have enough cards (our study included 21) to make use of a software to run it.

Once that was clear, we iterated on the cards. As our main question was format vs. progress, we didn’t want to lead the participants into grouping formats and progress together because of our choice of wording. We thought shuffling the words on the cards would avoid this and our first iteration used this pattern:

• Not Started – Audiobook – Downloaded
• Podcast Episode – In Progress – Not Downloaded
• Downloaded – Audio Summary – Finished

After talking with Temi, our Director of Design, we decided to test a second approach using a more explanatory wording to help the participants get into the context and move away from more technical language.

The second one used the following pattern:

• A finished podcast episode
• A podcast episode that I haven’t started yet
• An in-progress podcast episode that I’ve downloaded for offline listening

We ran a quick test with two people who both felt confused and frustrated with the first approach while they found the second one straightforward and enjoyable.

We decided to exclude “functions” like search and filters to focus on the hierarchy of the cards and probe with a user interview to learn more about their rationale behind their decisions.

Results analysis

We initially thought of running a similar analysis to the one presented in this article but because of the low number of cards and the tie in the results (5 progress vs. 5 format) we decided to focus on the qualitative data and cluster the main findings among participants.

Once collected, the data was analyzed to identify common trends. We analyzed qualitative information based on user comments and analyzed quantitative information based on:

• Which cards appeared together most often
• How often cards appeared in specific categories

The main clusters were either formats with progress as sub-clusters or progress with formats as sub-clusters with some variations in the ways the participants named the groups.

The main commonalities we identified among the participants helped us make decisions.

Examples of main commonalities:

• Finished content doesn’t have to be in the main list as participants rarely go back to it unless they want to reflect on their consumption and share with their friends
• Favoriting was useful for some participants to either share or revisit the content later
• Most people said they would like to be able to create their own lists either for listening in “hands not free” situations e.g. when cycling, or to create a topic list for themselves. Several participants mentioned they’d be interested in mixing the formats in their lists.
• Mood is an important factor as it often determines if people feel like listening to a shorter or a longer format, or which topic is interesting for them in the specific moment.
• Some people tend to base their choice on the amount of time they have available for listening.
• Several participants have storage problems and need to be mindful of the content they download or have in their apps.

Highlights and quotes from the test:

• Organization
• “I spend enough of my time trying to find files. Organization is about: what do I want to listen to, is there new content – it saves time and energy in making decisions.” — Carol
• “I don’t care how it’s delivered, I just want one ‘up next’ to take away the decision-making.” — Simone
• “Categories help to see what kind of content you have saved.” — Migena
• “Not really aware, the app does it for me.” — Wendy
• “I download a few episodes at a time and add it to a playlist.” — Tom
• Content in progress:
• Several participants mentioned they don’t finish reading or listening to everything.
• When content is stale for a few days or weeks, it’s likely they didn’t like the content and won’t resume.
• Few participants delete content from their library (fear of commitment) unless they have a storage problem and need to actively make room in the app.
• Several participants prioritized content they have in progress before starting a new title.

Next steps

An exploration phase started before Christmas, where we came up with different concepts and IA. This included a first round of iteration: speaking with the engineers early on to exclude any technical difficulties along with several design reviews to gather feedback from the team.

Two variations of the new proposed library redesign were tested at the end of January. Overall the two directions performed really well, the participants had no issues navigating through both libraries and could find the content they were looking for easily. We also wanted to test the information and cognitive load and were impressed to hear the testers describe the prototypes as calm, clean, simple and easy to use!

As I was analyzing the results of the tests, the priorities and focus of the bigger project changed slightly and I had to take a step back. The User Research team was preparing a diary study for the audiobooks and we decided we could benefit from gathering learning on original shows if they were added to the library by then. The mission team committed to add them to the app within two sprints and we slightly delayed the other library improvements.

After the official launch of audiobooks and new original shows, COVID-19 forced us to adapt to a remote life for a while and changed our priorities to work on features that would make more sense to both the business and our users. By then, users could already save episodes to their library and the team released two components unifying the list views in the same release as Dark Mode! This was an exciting project as we now have a consistent styling within the app and across platforms cleaning the inconsistent typography treatment for authors, titles and subtext, unifying the progress indicators for Blinks and using the same cover size.

We’re constantly improving the product and adding more features to the app. I’m very excited to gather some more insight on the way and to give more flexibility to our users. Being able to curate your own content and customize your experience the way you prefer makes you want to use the product even more. I hope everyone will be thrilled with the new updates!