By Guest Author: Rachel Rogers
Let’s say you’ve just committed to conducting a global Blind Study, but you aren’t exactly sure what it is going to entail. I mean there is really no industry standard for a “Blind Study”, right? Well the cool part is you get to make it up!
So I started with the first step of figuring out who should participate in the study. Followed quickly with handing that off to the really smart Sales Operations group to go find people that matched the criteria. We decided that we needed a mixture of Sales Representatives and Sales Managers.The criteria for either group was either excelled at Sales or good Salesforce User. I know you are saying, aren’t those the same? Well while we would all hope they were, it doesn’t always equate that way. I wasn’t interested in who fit which criteria, just that they were all represented.
So easy peasy right? Well, turns out you should add one more criteria: “agreeing to what the appropriate sample size”. When you don’t add that criteria, you end up with 26 people you need to conduct one hour sessions with. Then you throw in the fact that you committed to having them done within a week, make that the week after Dreamforce, ummm yeah…..
Well all of that madness aside, the other thing that needed planning was how the sessions were going to be conducted. Some groups like to take the approach of recording each session and then playing them back to get feedback, but let’s go collaborative. I decided to pull most of my team and my trusty Sales Operations counterparts into these meetings. They were going to be called “Observers”. Partly because I loved the Sci-Fi show Fringe, but mainly because this was going to be their job. They were not to interact with the users, simply observe and take copious notes on what they saw. They were split into three teams with specific functions:
Goal: Look for ways that things could be streamlined for the particular process the user is trying to accomplish.
Example: In the previous UI there were hoover links that took people directly to related lists, but in the new UI those are gone. Is there a simpler way to accomplish tasks by page layouts changes or is there process optimization to prevent them from needing to access particular related lists?
Goal: Identify which areas appear to be ‘natural/self-learned’ versus items that cause the users a harder time to identify.
Example: In the previous version tasks/activities were listed in a “to do” list area and now are listed in the “assistant”. Is this easy for people to understand or do they struggle to find where items are listed?
Goal: Identify any elements that are not working in the application that users hit technical roadblocks. Anything in this section will be an immediate fix for development.
The next thing was to write a script so that we could take away the variables in what we asked the users to do. We wanted to control as many variables as we could so that the results would not be swayed. The important thing to writing the script was to determine what functions we thought a user should be able to accomplish in the given time window while still allowing them time to ask questions. To do this, I went back to the Sales Process and thought about the most common functions this group would be requested to do on a daily/weekly basis. I also wanted to make sure that the script never instructed a user ‘how to’ do something. It was important that we gauge natural learning/transition patterns versus influenced instruction patterns. Think of this more as watching
Follow our blog for the next instalment of the (Salesforce) Lightning Chronicles