The Blind Study (Salesforce Lightning Chronicles)

By Guest Author: Rachel Rogers

Let’s say you’ve just committed to conducting a global Blind Study as part of your Salesforce Lightning migration plan, but you aren’t exactly sure what it is going to entail. I mean there is really no industry standard for a “Blind Study”, right? Well the cool part is you get to make it up!

So I started with the first step of figuring out who should participate in the study. Followed quickly with handing that off to the really smart Sales Operations group to go find people that matched the criteria. We decided that we needed a mixture of Sales Representatives and Sales Managers.The criteria for either group was either excelled at Sales or good Salesforce User. I know you are saying, aren’t those the same? Well while we would all hope they were, it doesn’t always equate that way. I wasn’t interested in who fit which criteria, just that they were all represented.

So easy peasy right? Well, turns out you should add one more criteria: “agreeing to what the appropriate sample size”. When you don’t add that criteria, you end up with 26 people you need to conduct one hour sessions with. Then you throw in the fact that you committed to having them done within a week, make that the week after Dreamforce, ummm yeah…..

Well all of that madness aside, the other thing that needed planning was how the sessions were going to be conducted. Some groups like to take the approach of recording each session and then playing them back to get feedback, but let’s go collaborative. I decided to pull most of my team and my trusty Sales Operations counterparts into these meetings. They were going to be called “Observers”. Partly because I loved the Sci-Fi show Fringe, but mainly because this was going to be their job. They were not to interact with the users, simply observe and take copious notes on what they saw. They were split into three teams with specific functions:

  • Process

Goal: Look for ways that things could be streamlined for the particular process the user is trying to accomplish.

Example: In the previous UI there were hoover links that took people directly to related lists, but in the new UI those are gone. Is there a simpler way to accomplish tasks by page layouts changes or is there process optimization to prevent them from needing to access particular related lists?

  • Concepts

Goal: Identify which areas appear to be ‘natural/self-learned’ versus items that cause the users a harder time to identify.

Example: In the previous version tasks/activities were listed in a “to do” list area and now are listed in the “assistant”. Is this easy for people to understand or do they struggle to find where items are listed?

  • Technical

Goal: Identify any elements that are not working in the application that users hit technical roadblocks. Anything in this section will be an immediate fix for development.

Example: The custom Javascript button on the Opportunity page is not displaying so the user is unable to complete the requested activity.

The next thing was to write a script so that we could take away the variables in what we asked the users to do. We wanted to control as many variables as we could so that the results would not be swayed. The important thing to writing the script was to determine what functions we thought a user should be able to accomplish in the given time window while still allowing them time to ask questions. To do this, I went back to the Sales Process and thought about the most common functions this group would be requested to do on a daily/weekly basis. I also wanted to make sure that the script never instructed a user ‘how to’ do something. It was important that we gauge natural learning/transition patterns versus influenced instruction patterns. Think of this more as watching a kid with Legos. They have all the pieces they need to build a box, but how they go about building it and the order in which they process through it could be completely different.

Our script focused on Standard Objects only. Navigating your Home Page, entering an Opportunity, navigating related lists, finding your Forecast, etc. The instructions went something like this:

“Let’s move forward to Opportunities. Navigate to the view that you typically use to review your opportunities on a daily basis.”

The purpose for that instruction was to see do they use List Views, do they just jump straight to an Opportunity, or do they use a report/dashboard? Based on how they self navigated, we had a talk track for each of the possible navigation points. The next talk track would have been something along the lines of:

  • If the user goes to a List View follow this track:
    • Notice on the top right you now have a series of icons.
  • If the user goes directly to an Opportunity follow this track:
    • Look to see if you can find the {Related List Name} on the Opportunity.
  • If the user navigates to a report/dashboard follow this track:
    • Can you navigate to an Opportunity on this report/dashboard that you need to update?

The other really important item is to make sure that the Lightning environment you are using shows no major red flags to users. You want to make sure you do a UI technical review that demonstrates that all of the functions are working. For example, you don’t want to show a user a cool new page that has a big VisualForce error. Then you are ready to roll!

Follow our blog for the next instalment of the (Salesforce) Lightning Chronicles: Blind Study – What Happened?


Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>