Material Metrics: research-backed validation for adopting Material Design (Google I/O ’18)
Articles Blog

Material Metrics: research-backed validation for adopting Material Design (Google I/O ’18)

August 10, 2019

afternoon, everybody. Hello. Thank you so much for
coming to this session. I’m Elizabeth
Churchill and I head up research for Material Design. And I’m sure you’ve been hearing
about a lot of the innovations that Material has brought
out recently here at I/O. And we’ve done a lot of research
in the background to back up much of that innovation. And what we’re going
to do today is just tell you a little bit about
some of that research. We’re also going to encourage
you to come and talk to us later, and we
will be in Dome F. Because we’re
interested in knowing how your integrating research
into your product and design system development. But before I get
going, I want to know how many people
out there consider themselves to be developers. Yay. Welcome. Thank you for coming. And how many consider
themselves to be designers? And you might have
multiple roles. Wonderful. This is great. Finally, researchers. Yay, research. Welcome, welcome, welcome. One of the things we want
to emphasize in this talk, and that Material is
extremely committed to, is that we believe the
tight collaboration between engineering
and development, between design and
research, really is the key to creating
great user experiences. And by user experiences I
mean the users of the things that you build, but
also you as users. Because in Material, as you will
see and you’ve been hearing, we build products for
you so that you can build great products for your users. So I just wanted to
go over a few things about Material, which you
may know, just to remind you. So the Material Design
system is open source. It’s adaptable, and it helps you
bring high quality experiences to your users. And our aim is to help you
make things that are beautiful, but that are usable and
useful and also, importantly, accessible. We’re very committed
to helping you do that. So Material consists
of components. And components are
things like buttons, things like text fields,
and things like tables. You’re familiar with that. And at this I/O we
have brought Material theming into the picture. And we’ve heard you
and we have responded because we want to make all of
these components customizable, so that you can adapt
them for your experience, so that you can have
your brand and your look and feel for your users all the
way through, systematically, from design to code. So that’s very much a key
part of our innovation. Material also consists of
patterns, patterns like account switching, accessing settings,
or handling empty states. All of this information, these
components, these themes, and these patterns,
roll up into guidance that you can see on
our site are spec. If you have not yet checked
out, please do so. Please go and have a look. Check out what we’ve got
there and send us feedback. Because a great
part of this talk is to remind you
that we can shape great experiences for people
with the help of folks like you. So we need your feedback. Material also, as
I mentioned, has a suite of products and tools
that are designed for you. One of them is here. This is Gallery,
and Sara is going to be talking about Gallery
in a bit of detail later. Gallery has also been
launched here I/O and, again, we
want your feedback. Gallery will allow you to take
those components, those themes, and that guidance and
turn it into reality. We’re really
building these tools to build this relationship
and this collaboration, the deep collaboration between
design and development. So just as a pause,
this is the outline of the kinds of research
we do in this team, in the Material
experience research team. The most granular
research is on components, and Michael is going
to be taking you through some of that. Here we look to make sure
the components really are beautiful and usable
and useful and accessible. The next level of research
is the product research on things like Gallery. And here again we use
interviews and observations, and we work with folks like
you to make these tools great. Sara’s going to be taking
you through some of that. Finally, we look at workflow. Where do all of these
tools fit into your lives as designers and developers? How do you pull these
together to accomplish the things you need? What is the bigger ecosystem of
the idea to design, to develop, to iterate workflow. And again, we do interviews,
we do behavioral studies, we observe folks in workplaces
like where you work. We do surveys to
try and understand where our tools fit in for you
now, how we can improve them, and to start to
think about where we need to go in the future
as the whole field explores and innovates. And Sara is going to
help you with that. So with that said, from our
studies with a few users to thousands of users, from
our close studies as we talk to you to our
surveys, we’re going to give you a bit of a
flavor of that today. And I would like to
hand off to Michael, who’s going to talk about
components research. [APPLAUSE] MICHAEL GILBERT: Well, thank
you for the introduction, Elizabeth. And once again,
welcome, everyone. My name is Michael and I lead
up research on Material System. So as Elizabeth mentioned,
we do research on everything that covers the minor details
of users’ experience, all the way up to the big picture. We look at everything from
how things look on the web or on your mobile device to
how those things come together to form an application or
a brand or an experience. For today, I’m
going to be focusing on two types of research
that we’ve done, both examining the components
of a design system. First I’ll go into some detail
on the top-down experimental work that we’ve
done, which looks into how the details
of design can really impact how your users
interact with your products. And second, I’ll go through some
of the bottom-up experiential research that we’ve
done, focusing on those same components, but
also aiming to understand how experiences come
together in our products and what those experiences
mean to our users. And by the end, I’m
hoping to give everyone a better understanding about
how we do some of the research here at Google so you can
take that back and do it within your own teams. But first, a little
background on what I mean when I say components. Elizabeth already
introduced some of the components, the
patterns, and the guidelines available in Material Design. Here we have three
examples of abstractions of those components. And just to introduce
them, on the left we have the Material FAB, or
the Floating Action Button. This was introduced in the first
iteration of Material Design and since has become one
of its more recognizable characteristics. In the center we have
the Material text field, and this provides a core means
for users to input content into applications. And on the right. material
Buttons, which provide a core means of interacting
with content in applications. And you probably are all
familiar with these already. But what I want to
call out here is not that these components
are necessarily unique or that there are
earth-shatteringly critical to any one
single interaction, but that these components are
a fundamental part of a larger experience. And because of that,
we want to make sure that we understand
these components at a fundamental level. But for now, we’re
going to focus just on this first one, the
Material text field. So this is the text field
that we started with. This was our baseline. This text field
performed well and it aligned with the
original Material vision. But there was an
opportunity to make sure that it worked even better, that
it worked across all devices, that it worked in both consumer
and enterprise applications. And when you look at
something like this, think of all the different
parts that have to come together to make a text field. There is the underline
the indicates where the field starts
and where it stops. There is the label that hints
at what might be typed in there. And there’s the text itself,
once you’ve entered it. And of course, there’s the
shape of the entire component, how each of those individual
parameters come together, and then how they’re
used on a page. So it’s clear that
even for a text field, there’s a lot going
on design-wise. But really, how many
options could there be? Well, it turns out, quite a lot. And as an aside, I’m
not expecting everyone to read through the
actual text on this slide. I just want to show
a few of the details that we’re considering here. So what we’re looking at
here is an exploration of label placement, as well
as some of the possible values and consideration
for label placement. But we also looked at
things like background fill. We looked at different
elements of style, different types of
contrast, and, of course, whether there should be
squared or rounded borders. In the end there were over
140 different variations that we looked at in a
couple different contexts. First, we looked
at simple forms, which had stubbed content
that was procedurally drawn so that we could test a
whole bunch of them. Second, we looked at real
quote, unquote “forms,” which were forms that we
actually took from the wild and inserted our
experimental text fields. And then third, we
looked at complex forms. And these are forms we
hope you would never encounter in nature. They’re unreasonably complex. But we created them to see how
each text field would perform in extreme circumstances. Would it stand out too much
or would it get lost entirely? And we’ll show examples
of that one in a minute. But first, what
does that look like? This is, ultimately,
a tool that we created that would create
each of those forms with each of those text
fields, randomly drawing the layout, the theme, and
each of the characteristics that we had identified. We then deployed these forms
and a set of large-scale surveys so that we could test
both the performance of every individual text field,
as well as get an idea of what users actually preferred. And on the right, you can
see the admin interface for this tool that we
built. And it’s simple, but it allows us to zero
in on the types of designs that we want to research. And more importantly, it
provides us with an artifact that we can share with the
designers and developers who are creating and building
these text fields. And maybe you’re curious
about the complex forms. Well, this is what I mean
when I say complex forms. We don’t condone
their use in the wild, but this is just so you know
the lengths designers will go to make sure that every
opportunity is explored. And they really are something. What we were able
to do, in the end, was to create a way for
people to interact with ideas rather than products. And to be clear, we
weren’t doing this research as a way of telling designers
what products should look like. We were doing research as a
way of working with designers to show them what
parts seem to work and what parts might
require caution, to provide them with the
tools that, ultimately, can allow us to more
effectively bring our users into the conversation. And this is one example of what
that conversation looks like. This is what the data actually
is from those text field experiments. And without going into
too many of the details, I want to show you
how we were looking at all the possible text fields,
every possible combination, without dropping any
of those possibilities because the data
says this or that. Each step that I’m
showing you here involved a process of
working with our team to explore the landscape
and to provide our designers and engineers with the
tools that they can use to create great products. And here, I’m referring to
research and the artifacts from research as
one of those tools. And here is the before and
after example of what just one text field might look
like, given this approach. Because ultimately
what we aim to do is to support design that is
more usable, more useful, more beautiful, and,
importantly, more customizable, especially with
today’s enhanced Material Design system and
the introduction of Material theming. We want to emphasize that the
components are customizable, and that within that
customize ability we’ve done the legwork to make
sure that those components will still work the way
that they should and, importantly, that they’ll
still look the way that you want them to. So that’s a quick look
at some of what we’ve done with the
Material text field, but let’s take a look at
the FAB and the button next. At the beginning, I
mentioned that we’d be looking at both the top-down
experimental research that we’d done as well as the bottom-up
experiential research. And this may look
familiar to many of you, this bottom-up research. It usually involves sitting
down with your users across a table,
sometimes in something like usability
studies, sometimes and things like interviews. And frequently it
looks a lot like just a normal conversation. But what we try to
do as researchers is to make sure that we’re
asking the right questions. And that’s not always, how
long does it take 10,000 users to click on this? Sometimes what we want
to know more about is how the users think and
feel about our products and about interactions. And because Material Design
is about the performance and the usability of components,
but it is also about that look and that feel. So next I’m going to show
three examples to give you a sense about how users
feel about these components and what that means
for our design. So this first example is
from a study participant who is interacting with
the new Material button. And she said, “When the boxes
are really bolded like that, it makes you feel like you’re
actually pressing a button.” And sometimes
something like this that happens in the
middle of these interviews will make you actually take
a step back and ask yourself, what is a button, really? And, really, what
should a button look like if we define it by how
it feels when we press it? Or what should a button look
like if it’s trustworthy? And these aren’t just strictly
questions for fun, either. There’s an entire
landscape of what buttons might look like, of what they
might be, of what is button-y. And part of the
research we get to do is to better understand
that landscape. And we get to do that by
talking with our users. And the second example is
from a study participant who was interacting with
the Google Material themed FAB shown here. And she said, “The last time
I looked at your email app, it didn’t look as
Googley as this.” And I love this because when
we actually talk to our users, we learn about more than
just the components. We learn about more than
just the interactions or the products. And in this example,
we can clearly see how design can impact brand. But we get a better sense, from
the user’s own perspective, about what seems to
jump out at them. And in the end, we can
speak more effectively about the meaningful
characteristics of brand with the designers
and the developers who are building these. And this last example is a
quote from a study participant who could use assistive
devices to see, but he was actually
legally blind. And he was interacting with the
Google Material themed extended FAB shown on the bottom. He said, “It actually jumps
out a little bit more, even without the color. It pops out. It just seems more obvious
what’s going on there.” And I think this is one
of the main strengths of actually sitting down
and talking with our users. And the same way we want
to explore how a hundred different text fields can
be implemented in a hundred different ways,
we want to see how these components
in our products are encountered by all our users. It’s a different way of looking
at the exact same landscape. To wrap things
up, I’ve shown you a brief look at how
we’ve approached a couple different
types of research. First, looking at the Material
text field from the top down, experimentally. And second, looking at the Fab
and the button from the bottom up. I’m hoping you can see some of
the similarities of these two approaches. On the one hand, we are
aiming to better understand a whole range of
possible expression to better support the brilliant
designers and engineers who are building Material Design. And on the other hand,
we’re aiming to better support our users to make
sure that each component is usable, is useful, is
beautiful, no matter how it’s used or by who. And last, again,
I’m hoping you might be able to take some
of the ideas we’ve talked about here to
be able to do research within your own teams. And with that, I’d like
to thank you for your time and welcome Sara
to the stage, who’s going to be talking a little
bit about designer and developer research. [APPLAUSE] SARA CAMBRIDGE:
Thank you, Michael. So as Michael said, I’m Sara. And I lead research
for Material product. And what we do is create
products and tools, resources for designers and
developers just like you. So Michael shared with you how
we do research with our end users so that we can
really understand how to make our
components most effective, but you are our end users, too. And designers and
developers is where I put my energy to
understanding our products and how you’re using
them to make them better. So what I’m going to talk
about is the other two types of research Elizabeth
mentioned, the workflow research and product research. And I’ll give you
examples of how products that have
launched here at I/O have been made better
through our research. So let’s start with
workflow research. This gives us a really
big, broad picture for how people creating
digital tools, people like you, how you’re doing your
work day in and day out. Usually, how this looks
as I’ll interview, ideally, designers and
developers who work together. I’ll go to your office. I’ll talk to you about
how you’re working. I want to know all
the tools you’re using, who you’re
collaborating with, and, especially, anything that’s
a problem in your workflow. And our goal with
this kind of research is to get a big picture
of the bigger problems that affect a lot
of people so we can prioritize
building tools that solve those problems for you. So this example is
from workflow research we did with designers and
developers that work together. Like I said, we went
to their offices. And one of the
things we had them do is take us through a recent
project from beginning to end. We had them walk us through
every part of the project, who they worked with, what
the tools they were using, and especially anything
that was a struggle. So here’s a couple
of examples of how people think about their work. So you can see people
look at their work in really different ways. Whenever I do this
kind of research, I want my developer, designer,
and product manager colleagues working on those
products to come with me. Because every one
hears different things, and different things pop out
at them during the interviews. So we always do a
debrief where we compare notes on what
we heard and what was interesting or surprising. And because this
research was really focused on understanding the
UX design process itself, we were going to be doing a
timeline of a typical UX design project. And so I had my
colleagues categorizing the different phases that
they heard the people we talked to went through. So let me show you
what this turned into. This is a UX design workflow. It’s also known as a journey
map or experience map. But what we ended up
with was six phases for the design process– discovery, initial design,
iterating on the design, validating, beginning
building, and then the final implementation. And I’ll explain
the structure first, and then I’ll come around
and fill in some detail. So below that, in
the green to red, are a single positive
and a negative key moment in each phase. And those were from
stories we literally heard from the participants
we talked to that were either really meaningful or
were a pattern we heard several people talk about. And then at the bottom, we
had the tasks, pain points, and tools. And this gave us a
really rich range of experiences that we heard
different people using. Now, I took that out to just
simplify it to show you, but that whole thing
is a big poster that hangs in our office. So for instance, thinking
about the key moments, often we heard people
get really excited when they’re starting a new project. There’s a lot of
promise and potential. But they quickly get overwhelmed
by scope or scope creep, so that becomes a
negative moment. And as you might
imagine, there’s a lot of interplay between
designers and developers here. The first one is when, in
the initial design phase, when a designer is really
excited about an idea they have and they have to go
to their developer, and the developer says to
them, we cannot build this. The technical constraints
make this impossible. So has anybody here ever had
one of those conversations? OK, all right. That’s good that we’re
resonating with your experience as well. But of course, there’s
positive interactions between designers and developers
throughout this process. We definitely have heard
that designers do not enjoy making design specs
for handing a project off to a developer. But when that moment comes where
they get to hand off, formally, to a developer, that’s usually
a milestone, a really positive milestone in a project. So that’s positive. And it can go south again,
because what happens often is that
developers come back later with unexpected
problems due to legacy code and the designers
have to step back in, often after they’ve already
moved on to another project. So again, is this resonating? Anybody had this happen? OK. All right. So the whole point of doing
this kind of really deep dive into understanding
the practices, again, is to surface
those problems that affect a lot of people so we
can prioritize building them. And I just want to take
a minute and define– I’ve mentioned
design specs and I’ll talk about it more in a minute. But when I say design
specs, what I mean are the margins, guidelines,
colors, fonts, and often even behaviors that designers give to
developers when they’re getting ready to implement a design. So another thing that
came out of that research is pain points of
designers and developers. And I’ll go through
these with you. So the key pain points
we heard from designers are having to aggregate
feedback from a lot of different stakeholders. Often the feedback is
living in different tools. Having to prototype motion,
but not having time to do it. Having to keep track of
different project resources that they’re going to need at
different phases of the design process that often are
living in different places. Now, remember this
because I’ll show you a solution that we came up
with for this in Gallery. Creating design
specs, as I mentioned, and then having to share
design assets across their team and keep them consistent. Top pain points we
heard for developers was having to solve
for edge cases, and often responsiveness
from a singular happy path that they’re getting
from their designer, getting incomplete design specs
or getting ones that are wrong or not getting them at all. I’ve absolutely I’ve heard
that one a lot as well. When you have to
create a new component, trying to figure out,
am I going to customize something that’s
existing or build something new from scratch. Getting design changes
late after you’ve started working,
leading to rework. And finally, having
a lack of context of what the intent was
with the design, so not knowing what really matters. So you’ll notice with
this kind of list that both designers and
developers, both found design specs as a real pain point. So it was this kind
of research that let us realize what a problem
this is for both roles, and let us prioritize building
a tool that solves this problem. And that project, as Elizabeth
mentioned, is Gallery. And I’m going to tell you more
about Gallery in a minute. But before I do, I’m
going to tell you a little bit about the other
type of research I mentioned, product research. And this is really
just a big bucket for any time we show
you something that we want to get your feedback on. It may be a really
early stage idea that we just want
to validate and make sure the concept is
sound, or maybe it’s a refinement of a product we’ve
already invested a lot of time in and we just want to
make sure that it’s really functioning well for
how you want to use it. So a lot of people
call this usability, but for us, this
type of research is doing so much more
than just telling us if a product is usable. So also, let me walk you
through a little bit about how I conduct research when
I’m doing product research with designers and developers. So this is one of our
research labs on the left. Usually I’ll bring
somebody into our lab. I’m meeting with
them one on one. It’s streaming to a room
that my designer, developer, and product manager
colleagues are sitting in, like the one on the left. And I’ll walk you through some
kind of prototype or beta tool, and getting your
feedback as we go along, like what do you like,
what’s confusing, and especially,
at the end, I want to know how does this fit
into your existing workflow. Is this going to solve
any problems for you? And then after all that
is done, go back in, talk to my colleagues,
and we figure out what was really
interesting here. What can we take from
this that we can use to make our products better? So now I’ve told
you a little bit about how this process works. Let me walk you through
an example of something we learned when we were
doing research with Gallery and how that made it better
for the final product. So has anybody seen a
demo of Gallery in the– OK, just a couple folks. And let me walk you through. So this is a project
page in Gallery. And also let me just
give an overview. Gallery is an online
collaborative tool that lets designers share their
designs with other colleagues, get feedback. And it also has a
design specs feature where they can get the
metadata from the sketch goes into an online
way for developers to get the code and the values. And then the developers
also have an access to Material components
in that Gallery feature, if they’re using the
Material components when they’re designing. And it’s being demoed
in Dome F over there, if you want to check it out. So we’re in a project
called Posivibes. There’s a little description
of the project at the top. The colored rectangles
are a way for designers to collect the
links they’re going to need at different
stages of the project. So as I mentioned,
we were solving one of the problems we uncovered
in the research with that one. And then, below
that are basically folders and people can organize
them however they want. And in this case, they’re
organized by the device that it’s being designed for. So let’s say you clicked into
the tablet view from here. You would see all the screens
better in that tablet. And if you clicked into
the first screen, then you’re going to see that
screen in more detail. Now, at the yellow
box at the top you can see all the different
things you can do from here. You can comment, inspect,
you can share it. But what’s happening
here is someone has rolled over the type at
the top, and you can see it’s got the typeface, the
weight, the color. All that stuff is right there
for the developer to access. So that’s how it looks now. And let me show you something
that we were getting feedback on a couple of months ago when
we did some research on it. So this looks a
little bit different because it’s the older
mock, but you can see it’s basically the same thing. So in the study, what we did is
we were talking to designers. We had them use our new
Material theme editor, which I hope you’ve heard about
the Material plug-in and we have for Sketch. And that gives them access to
all these pre-built Material components where they can
customize for their own brand. So that the one in the green
bar, the green bottom app bar, they had customized that in
Sketch and changed the color and changed one of
the icons in there. And then they had
uploaded that to Gallery, where they were looking
at what they would be sharing with their developers. So we were wanting to know,
what do you think of this? And what we heard from
them, as you can see here, when you roll over that bottom
app bar, what you’re getting is the link to the code
and the specs for that. But you’re not getting any
more granular information beyond the size of
that entire bar. And what we heard
designers saying is like, well, I want to see
what that shade of green that I changed it to was. I want to make sure it’s
my right brand color. And I want to see
what that icon was. I want to make sure it’s
the right icon in there. So what we heard is that they
wanted a lot more granular data for the Material components
that we were showing them in Gallery. And we had it. We just didn’t put it in there. So this was a really important
thing for us to know. So this is the final version
that’s been released. So as you can see, you’ve
still got that bottom app bar. But when you roll over any
single component inside it, you get that really
granular data for that particular component,
in this case, the size of that diamond-shaped FAB. So this is just one
of many examples of how the research has made
our products launching here at I/O better. Another one is the
Material I/O home page. So we in the same research
I’m talking about, we also got feedback
on an earlier version of the home page design. And we heard from
designers that they really like to get overview
videos of any time there’s a new evolution of
something, so they can get a quick sense
of what’s changed. And we had that, but I was at
the bottom of a really long page and nobody found
it before they were able to click on the video. So what we did is in the website
that went live a couple of days ago, right underneath that fold,
that video is really prominent. So we made it
really easy for you all to get a quick overview
of what’s new in Material. So what I’ve just gone
through with you has given you some examples of how we use
workflow research to identify the big problems that we want
to put our time into solving, and how product research
improves the product, no matter what stage it’s
in, and how we use products that we will be
releasing at I/O, how we use research on these
products to make them better. And if you want to
go check out Gallery, go to materialio/tools. All right. I’m going to hand it
back to Elizabeth now. ELIZABETH CHURCHILL:
Thank you so much, Sara. Thank you, everybody. So just a little bit of
a wrap up and a reminder of the invitation to
visit us in Dome F. And so what we’ve shown
you today is just a flavor of the ways we’re
using traditional methods for research, as well
as inventing new methods to understand how the components
work for everyday users, in terms of beauty
and aesthetics, in terms of usability, in
terms of sort of usefulness, but also accessibility. And Sara has shared
how important it is for you to send us
feedback on our products. Because we do want
to work with you to make sure that
your lives are easier in this pretty difficult
workflow process from idea to product that really works. Another thing I
want to let you know is that we know that many of
you don’t have research teams. We’re lucky enough to have one. And so we think of ourselves
as the research system that aligns with the design
system and the engineering system. And we have committed to
writing up some of the methods that we’re using and
inventing, and we will be sharing that along with
the other guidance and spec, and sharing how
these methods can help you understand how our
tools work for your context. Because as Michael
said very clearly, your context might be slightly
different from our own. So we will be sharing
more of that with you over the next year. And so, finally, I
hope you take away how we’re bringing together
design and development and engineering and research,
and how we’re really treating this as a conversation,
and a conversation that needs to be a rich one. And so with that said, thank
you again for your attention. Please come and join
us in a Dome F– Tent F, I should say,
at design accessibility. And again, thank you
for your attention. Thank you for coming. [APPLAUSE] [MUSIC PLAYING]

Only registered users can comment.

  1. Well done presentation. Really enjoyed Sara Cambridge's explanation of how to use Material in the research process.

  2. Those new text fields are probably one of the worst design things I've ever seen I mean I understand what they trying to do there but it really doesn't live up to my material design expectations. I'm disappointed appointed in you Google. Hopefully they just keep this

  3. Very frustrating editing. Many slides are never shown, even right after the speaker announces that we should look at the before and after images on the slide… That we never get to see.

    Otherwise, I felt like this was too high level. It would have rather seen more of the details in one area, instead of trying to briefly cover it all in one session.

Leave a Reply

Your email address will not be published. Required fields are marked *