User-testing Gutenberg

At Yoast, we organize a company-wide user testing session each month. In these sessions, we ask our employees to take a look at a product or feature. On October 12th we focused on user-testing Gutenberg. We did this to gain a better understanding of how different users work with Gutenberg. This knowledge is sure to help us in our own contributions to the Gutenberg project. Today we’re publishing the test results for reference.

We will discuss these outcomes with the Gutenberg team and create issues on the Gutenberg repo for the things that need fixing. We hope others will be inspired by this document to organize their own user tests around Gutenberg. Fixing some of these usability issues isn’t even that difficult, but they have to be found in the first place. We have to keep looking. And when we do, we can make sure Gutenberg becomes the best web content editor around!

What is WordPress Gutenberg?

Gutenberg is the new content editor that is supposed to debut in WordPress 5.0. This new editor is a complete departure from the old WYSIWYG editor that is now available in WordPress. Gutenberg introduces the concept of ‘blocks’ of content that can contain almost anything and be arranged in any order. While we’re fairly skeptical about certain aspects of Gutenberg and the way it is developed, we’re also excited about the new era that this editor will bring. We’re actively looking at how to integrate Yoast SEO into Gutenberg.

Table of Contents

• 1 Setup
• 2 The results
  • 2.1 Interface
  • 2.2 Content editing
  • 2.3 Media
  • 2.4 Mobile
  • 2.5 Positive remarks
• 3 Conclusion

1 Setup

We gathered 40 people of diverse backgrounds and technical experience, from absolute beginners to experts and WordPress contributors, and divided them up into teams of two, who were tasked with recreating this post from The Next Web in Gutenberg v1.4.0. We chose that post because it has a bit of everything; It contains straight-up text and headings, links, images, galleries, quotes, embedded tweets and videos.

2 The results

Below are their feedback points, grouped per subject and edited for clarity. We’ve also tried to list possible solutions for most of them, and mention any related issues from the Gutenberg repo (that we could find).

2.1 Interface

• The preview button was hard to find. The eye icon without text or a button shape does not immediately read as the preview function.
  Possible solution: give the top bar icons a proper button shape. Somewhat discussed here.
• Text mode was hard to find, hidden behind an ellipsis dropdown in the corner of the screen. Once people get used to Edit HTML, this will be less of an issue. But that function is currently also hidden behind a different ellipsis menu.
• Some people complained that they couldn’t easily convert content to a different type, indicating the block switcher was sometimes missed, probably because it is not a familiar concept.
  Possible solution: change dropdown from showing what the block currently is to the text Convert into.
• It’s not possible to move blocks by drag-and-drop, only by arrow buttons. It’s not clear that you can also bulk-select blocks and cut/copy-paste them.
  Possible solution: turn the arrow buttons into a grabber.
• The lack of a button shape and obvious hover on the Inserter led to people not identifying it as a button. They thought the flyout shortcuts were the only way to add blocks at first.
  Possible solution: Dropbox Paper does a great job at making the inserter an obvious button. Related to the discussion in the first point.
• People also often used the Inserter in the content first before discovering it in the top bar. Could be solved by the first point too.
• The behavior of the Inserter is sometimes unpredictable. Having a block selected and using the Inserter places that block under the selected block. But the selected block can sometimes be way offscreen, making it confusing where exactly something just happened.
  Possible solution: display the Inserter between blocks. This has been implemented via this PR.
• Block settings don’t automatically show when you select a block. It requires a click on the cog icon (which is under an ellipsis menu now). Lots of steps to get to the options you want. And if they’re open, the icon seems to do nothing.
  Possible solution: selecting a block could switch the sidebar to the Block tab, or the cog should be more accessible. Discussion about this took place in this Github issue.
• Cannot edit the font or color of a selected sentence in a block. This is apparently up for consideration in v2 of Gutenberg; the discussion was closed.
• There are differences between the editor (backend) and the preview (frontend). Obviously font and size and color, that’s inherent to the theme, but also list indentation and button alignment, whitespace here and there, code block style, and definitely the table block.
• Removing blocks is a chore, now that the trash icon is hidden behind the ellipsis menu.
• The floating toolbar is sometimes distracting, it moves inconsistently, and has inconsistent contents.
  Possible solution: Dock the toolbar at the top.
• In general, working with blocks can feel like you’re building a post with legos, instead of writing copy. The blocks are very prominent. It takes some of the dynamism out of chopping up your text as you’re drafting, it becomes more rigid. Some efforts to improve this are being done here, and get to the discussion around having a dedicated Edit mode.

2.2 Content editing

• The Title block doesn’t have color options or any options at all. It’s been discussed whether it should even be there or not, but the fact that it is a ‘special’ block should not exclude it from having a few options.
• The permalink is not visible initially as you type in the Title block, it only shows up later.
  Possible solution: While it’s nice not to get that big permalink popup when you just start writing, maybe it can appear in a more understated way, fading in as a small text above or below the title. Discussed here, has some REST API problems it seems. Putting it in the sidebar is an option too.
• Editing the permalink is impossible. Same discussion as above.
  Obvious solution: it needs to be editable.
• Pressing Return in a Title block does not move to the first paragraph block. This issue has been fixed in v1.5.
• Why isn’t Featured Image a block?
  Possible solution: make the cover image block be the featured image.
• The toolbar of a Heading block can only go up to an H4. The sidebar offers H5 and H6. That’s strange. Possible solutions: include H5 and H6 in the toolbar, or only allow the heading level selection in the sidebar.
• Pasting multiple paragraphs does not put the caret at the end but the beginning.
  Obvious solution: it should.
• After having added a link, if you click the options icon and press the Open in new window toggle, it closes the menu, robbing you of the ability to see the result of your action (the toggle turning on).
  Obvious solution: it should stay open.
• The options menu of the link popup can be open at the same time as editing a link (the two menus overlap).
  Obvious solution: when you edit a link, all other related popups in that menu should be closed.
• Quotes with paragraphs break into multiple blocks.
  Possible solution: don’t treat a <p> tag in a quote block as a ‘block breaker.’
• Changing paragraph size can make it hard to tell what is body text and what is a heading – for example, a paragraph set to 26px bold looks identical to an H2.
  Possible solutions: 1. Live with it 2. Don’t allow font size changes (should depend on the theme maybe) 3. Find an alternative way to display what the block type is, even when the block UI is not visible.
• Text column block does not have the same options are a regular text block (text settings, colors)
  Obvious solutions: add those options to the column block, or add the column options to the paragraph block (a better idea maybe).
• Text columns cannot have individual headings or pictures. This is a tricky one, as it gets to the matter of column/grid layouts. For sure needed to fulfill any page builder ambitions, maybe not in scope for v1. But regardless, great designs have been posted here already.
• Pasting into an empty heading block creates a new block below it and leaves the one you already made empty.
  Possible solution: tweak the paste-conversion system also to overwrite the selected block (if it is empty).
• If you select Edit HTML on a paragraph block and then type in an image tag, you get a warning that the block was modified externally. Possible solution: maybe Gutenberg could first try to parse the HTML (since in this case, it was valid) before deciding to quarantine the entire block.
• If a block is locked by external changes, no option sounds like revert/discard. Overwrite seems to come close, but what does it mean, with what will it be overwritten?
  Possible solution: remember the state of the block when you click Edit HTML, so any incompatible changes can indeed be discarded properly. In our case, pressing Overwrite cleared the entire block.

2.3 Media

• Images that are less wide than the content area are not centered when you press the center alignment button. Their block is, but the image inside it is aligned to the left.
  Possible solutions: align it to the middle of the block, or add an extra button specifically to center the image (as you could also want the image to be left-aligned but not next to text).
• Dragging multiple images into an image block does not create a gallery.
• Dragging images into a gallery block does not work. It creates a new image block with that image above or below the gallery block.
• Changing an image to a gallery by dragging in more images does not work.
  Possible solutions: maybe a + button appears next to the first image, or add a block switcher to the image block to make the transformation easier.
• Images added by drag-and-drop are not uploaded to the Media Library.
  Obvious solution: they should be. Noted here.
• Changing the order of the gallery block would be nicer with drag-and-drop, instead of having to click Edit > Create new gallery > then change the order and click Insert new gallery, which feels like it would create a second gallery block.
  Possible solution: if the aforementioned grid layout was possible, this might not even be necessary, you can just chop and screw a gallery with some loose image blocks.
• “Embeds” is maybe not a very clear term if you’re looking for social media. Maybe “(Add) Media” or something less based on the technical term and more in the human need.
• Insert from URL is gone from the Media Library. Discussed here.
• Pasting an image URL in a paragraph block converts it to an image, although it remains a paragraph block??
  Possible solutions: just paste the URL as text, or convert the block to an image block.
• If you put an image in an HTML block and click on the image, the image options popup overlaps with the HTML block toolbar.
  Possible solution: center the image options pop up on the image.

2.4 Mobile

• The top bar looks messy, lots of different button styles and icons mixed in together.
  Possible solutions: rearrange the icons in a more calming manner, or restyle the buttons to look consistent here. Somewhat related to this discussion.
• The Inserter is very very hard to open. It zooms the page when it opens, which is weird. And sometimes it snaps to the block you were editing and the inserter menu expands upwards outside of the screen, so you don’t even see it. And other times it expands downwards but isn’t centered, with half of it outside the screen. This also happens with the inserter at the bottom, which you WOULD want to expand upwards. Whether it expands upwards or downwards is strangely consistent though (every other tap on the inserter button). Discussed here.
• When selecting something to make a link, the native iOS toolbar pops up over the Gutenberg toolbar. Related: this issue.
• Selecting a block is disorienting. It’s not clear when it is selected and what all the UI is that pops up suddenly.
• The block options are hard to access. Pressing the More button opens a small menu that itself has another more button, which expands downwards, hiding the Trashcan and HTML buttons so you cannot delete any blocks. Noted here, and fixed in v1.5.
• Tapping the More button shows the tooltip, which stays in a fixed position if you then scroll down.
  Possible solution: maybe just forget about tooltip on mobile, they rely on a hover, which doesn’t exist on mobile. Or only show the tooltips when you hold down on a button or something, a separate intent. Seems improved in v1.5.
• The keyboard pops up sometimes unwanted. Maybe you just wanted to edit some block settings or use the toolbar. Might just be a side-effect of mobile browsers. Seems improved in v1.5.
• Scrolling is not smooth; it doesn’t keep moving once you let go after a swipe.
  Obvious solution: it should.

2.5 Positive remarks

• Gutenberg looks clean and modern.
• Uploading images by drag-and-drop, no media library required, is nice.
• Thinking about your post structure and reordering content is fun.
• Easy social media implementation.
• Great that it remembers recently-used blocks.
• Easy for beginners.
• Pasting content (HTML, Markdown) converts to blocks well.
• Being able to switch a block to a different type quickly is convenient.
• The inspector works well on mobile.

3 Conclusion

As you can see, there is still a lot to gain regarding user experience. But, like G.I. Joe once said, knowing is half the battle. If we identify these issues now and come together to think about solutions, we can make sure the first version of Gutenberg that gets merged into core will be the best it can be. It won’t be perfect because it can’t do everything all at once, but in terms of content editing, we’re starting to get close to a solid v1.