SaaS Product Engagement - A Simple (Yet Genius) Strategy by Sherlock. Featured image for blog post.

Derek Skaletsky isn’t new to SaaS. Formerly, he founded Knowtify and Traackr. His latest venture? Helping SaaS businesses align their Sales and Customer Success teams with a singular product engagement scoring system.

Derek Skaletsky

Image Source: LinkedIn

We had the luck to talk to Derek a couple of weeks ago, in an interview he gave to our Founder, Aggelos. The two of them discussed go-to-market strategy (GTM Strategy), product engagement, and a methodology that Derek invented and called the “Scoring Methodology.”

This methodology is the foundation behind Derek’s latest startup, Sherlock. If you want to learn more about Derek’s methodology, how Sherlock helps SaaS companies better understand their users and what is Derek’s approach to product engagement, keep reading.

The Pain point With Data

“One of the biggest frustrations I’ve always had, at a very high level but also operationally when trying to manage sales, customer support teams and other departments, is knowing how important product engagement is,  in general, to the  SaaS business model, to our go-to-market strategy and the optimisation of our value proposition.” This is what Derek told us at the beginning of our interview.

As Derek told us, “you need to keep them engaged with a product in order to get them to convert. If they do convert, you need to keep them engaged in order to keep them—in order to retain them. It might sound like a riddle but is the secret behind a successful product launch and new product strategy”

Product Data

Image Source: Slide from Derek’s Presentation for the PLG Summit

In our experience, many SaaS businesses fail to identify a relationship between product engagement and actions like converting free users to paid ones which are critical for your GTM strategy. As Derek told us, he always found himself asking his team, “what are (our accounts) doing? Where are they at? Are they engaged? Are they using the product?”

But, unfortunately, Derek couldn’t get the answers he needed since “no one had access to this data because this product engagement data was always kind of stuck in some analytics tool with the product team that really never got used—and even if they did, they weren’t built to give the kind of actionable insights that the operational teams need.”

According to Derek, that was causing a major problem to sales and customer support teams, since that data was essential to them on an operational level. As we will see later, this causes a big problem not only in sales and customer support, but in departments like the marketing teams as well.

The Solution?

What could be the solution to that inefficiency of keeping all product engagement data in one place? According to Derek, “I’ve hacked systems for this many times. I’ve built internal systems. Had the engineering team build sequel queries that we could access and different tools and it just, you know, you build an admin interface and try and get some of that in there.”

Is that enough though? Is building internal systems enough to cover a fundamental need such as measuring product engagement? As Derek shared with us, “it’s never a priority to actually update that admin interface and make it better and better for the operational team. So it’s always been a challenge.”

Product Engagement

Image Source: Slide from Derek’s Presentation for the PLG Summit

This challenge—which was (and still is) causing a big frustration to the sales and customer support teams—was what lead Derek look for ways to “bring this product engagement data to those operational teams in a way that they could use it easily to help them make decisions and take actions and just bring clarity to their day to day work.

Of course, the fact that tools like Segment existed at that time, helped bring the whole idea to life. As Derek remembers, “Segment coming along and gaining huge adoption was a big help. I don’t think this kind of thing could exist if Segment wasn’t in place.”

Segment Homepage

Image Source: Segment

All that was enough for Derek to start his current company: Sherlock. As Derek told us, “we wanted to create a way to score that product engagement data; to what we call, quantify it. So that you could easily apply some kind of scoring model to your users and, in SaaS most importantly, to your accounts. So that you can quickly understand the engagement level of your users and your accounts which is critical not only when optimising your product but also when building a marketing plan or marketing strategy for existing targets markets or even a new market..”

Product Engagement Scoring Engine

Image Source: Slide from Derek’s Presentation for the PLG Summit

Derek continues saying that “it had to be a scoring model that our target markets could understand very quickly and easily. So that was the basis for the scoring model that we built.” That gave birth to Sherlock, and to the Scoring Methodology.

The Scoring Methodology is essentially a way of measuring product engagement to help the sales team and customer support teams align towards common action plan, that come up as a result of understanding users better.

Sherlock Score

Image Source: Sherlock

5 Steps for Scoring Product Engagement

Step 1: Define engagement for your product

According to Derek, the Scoring Methodology has five steps. “In step one, you define engagement for your product. This means that you are  tracking all the events, all the things that happen in your product. But every product is unique. So the first thing you need to do is really sit down and say what my key events for engagement are.”

Identifying your key events is the first step towards measuring your product engagement. How many such events you should use? As Derek told us, “that could be anywhere from 10 to 20 (maybe 30 events) that are really key to engagement with your product.”

Let’s illustrate this with a simple example: “For a B2B sharing doc, these events can be a document created, shared, a team member added, comments made or edits made on a doc.” As you can imagine, for different products, you’ll have different events.

As Derek told us, “for a team chat app, you’ll have a different set of engagement events that are important for your product. If you’re an accounting software you might have company created, account added, category created, expenses uploaded, bank integrated.”

Degine Engagement for Your Product

Image Source: Slide from Derek’s Presentation for the PLG Summit

Step 2: Weigh key events

In the second step of the Scoring Methodology, you have to weigh events based on their importance. According to Derek, “not all events are created equal or are of equal importance to engagement with your product. So you want to give them weight.”

Now you may be wondering: what is the weighing system you can use to achieve that? According to Derek, “you can have a weighing system between 1 and 100 or 1 and 10. This is what the system that we’ve built does: it scores events between on a 1 to 10 scale. Thus, each event has a weight based on how important it is to product engagement.”

Weigh Key Events

Image Source: Slide from Derek’s Presentation for the PLG Summit

Step 3: Count events for every user

In the third step, you need to score each event for each user. What this means is that “you count the number of times users did each of your key events in a period. This period is what we call your engagement period.

When asked about how long the engagement period should be, Derek replied that “the engagement period can be daily, weekly, or monthly. Generally, SaaS are weekly engagement products. Thus, most people use a seven day or weekly engagement period.”

Count Events for Every User

Image Source: Slide from Derek’s Presentation for the PLG Summit

“When you’ve gone through all your key events, you’ll count the number of times every user did that event. You’ll apply your weight to that. And then you’ll get an individual score for each event, which is simply the number of times they did that event times the weight of that event.”

That’s when the magic starts to happen since “then you can see on the right side there, a score assigned for each event. Last but not least, you need to add up those scores, and you’ll get a total score for that user for that period. This will give you what we essentially call a raw engagement score for that user, for that period.”

Count Events for Every User Raw Score

Image Source: Slide from Derek’s Presentation for the PLG Summit

Step 4: Normalize scores for context

So far so good, but how all teams involved in that process understand what each score really means? As Derek told us, “the problem is that you want this score to be consumable. To be easily understood by your operational teams and everyone in the organization. So you need to apply a normalization—what we call giving that score context.

Normalization is essential since the raw scores assigned to each user are not contextual. Thus, “we need to apply some kind of normalization to those raw scores so that you can get them into a 0 to 100 range. For us, using a 0 to 100 is the best way to go, since it’s super-easy for everybody to understand.”

Normalize Scores

Image Source: Slide from Derek’s Presentation for the PLG Summit

Step 5: Score at the account level

The only problem with this is that most SaaS businesses sell to accounts, which means that normalization on a user-level doesn’t work for them. Thus, in this final step, you need to apply normalization to your scores on an account level. Which is essential, since scores in a user level are “meaningless because every SAAS business is account-based.”

Score at the Account Level

Image Source: Slide from Derek’s Presentation for the PLG Summit

How can you do that? According to Derek, “just like you do it for users—if you want to get it to account based—you apply the exact same thing, only you aggregate up all the events of all the users on an account level. And you create the same kind of table, the same scoring methodology, the same normalization, and you now have got your accounts right and scored.”

Score at the Account Level 2

Image Source: Slide from Derek’s Presentation for the PLG Summit

The Importance of Product Engagement for SaaS Businesses

The Scoring Methodology helps you connect product engagement with user actions that happen on a product level. Moreover, giving scores to your users and accounts can help you dive a bit deeper in understanding metrics like retention when building your action plan.

According to Derek, this methodology can have a severe impact on retention since ”we always associate product engagement with retention. Users need to be engaged to stick around. So the first prominent place where this kind of scoring model can apply is retention. The post-sale retention, churn risk management, expansion opportunity.”

Is this correct, though? Is it right to connect product engagement only with retention? Or should we connect it with other actions like conversion? As Derek mentioned, “our argument is that product engagement is a fundamental metric in a SAAS model. It just simply is.”

And he continued saying that “if people don’t use the product, they don’t buy the product. They don’t stick with the product. You can’t figure out who to market to, and you can’t really find product-market fit until you understand who is most engaged with this product.”

The importance of product engagement in the sales team

As we mentioned earlier, product engagement is also essential on a sales team level. The way people buy products and services online has changed. This is happening because “there are very few products that don’t offer an option to try them before you buy these days.”

As Derek was deploying his thoughts on the importance of product engagement in the sales strategy, he told us, “whether it’s a premium model or a free-trial model—even an enterprise product that has a demo only sign-up flow—is offering some kind of pilot before purchase is made. Thus, understanding engagement during that trial period is essential for an effective sales strategy process to actually try to get those people to convert.”

Product engagement as a KPI

As we’ve stated repeatedly, product engagement is essential for functions like customer success, sales, and also for actions like retention and conversion. This makes product engagement something more important than just another KPI.

In fact, according to Derek, “once you are scoring all you users and potential customers, you can roll that up into a total product engagement score and see that change over time. And now you can start to track, if total engagement is going up or down, pre and post releasing features and making changes on a product level. Product engagement is a fundamental KPI for a SAAS business. A SaaS business can’t survive without product engagement.”

Measure Scores Over Time

Image Source: Slide from Derek’s Presentation for the PLG Summit

What happens when engagement drops?

A logical question arises here: How can sales and customer support use the score changes to keep their potential customers engaged; in other words, happy with the product? As Derek shared with us, “you can create custom Slack alerts based on any of the metrics that you track.”

And he continued saying that “we’ve noticed that most of our customers have at least one alert around engagement drops. So, if a big account shows an engagement drop of 10%, it creates an alert in Slack in the customer support channel, and that’s something to investigate.”

That doesn’t mean that you have to be obsessed with alerts when something bad happens. As Derek explained to us, “it doesn’t mean you have to drop everything and go fly out and visit them, but it definitely highlights what you need to do today; or what you need to think about this week.”

Optimize Customer Success Processes

Image Source: Slide from Derek’s Presentation for the PLG Summit

Something vital is to take actions not only when engagement drops—when something doesn’t go as expected, but also when things are going well for an account. In Derek’s experience, “I’ve sat through many of a sales management one-on-ones and customer support management one-on-ones where you’re talking to a particular salesperson or customer support rep about how their accounts are doing, how their leads are doing.”

Even though the responses varied, Derek identified a pattern in many of the responses that he received: “It’s just always, oh you know, well they’re doing great, I think they’re doing good.” However, this I think shows that there isn’t clarity here; this is why scoring based on product engagement is essential.

How to use the Scoring Methodology in marketing

The idea that this process of scoring users based on product engagement sounds really appealing and could possibly allow other departments—like marketing—make better decisions, based on the data they already have.

According to Derek, “I’d love to see it in marketing. I love to see smart marketers look at the engagement of existing accounts as a way to attract better leads.” If you think about it, marketing could indeed use product engagement as a way to understand which users get the most value out of the product.

Optimize Marketing Programs

Image Source: Slide from Derek’s Presentation for the PLG Summit

This data could later be used as a way to get better leads or create case studies around your use cases and target personas. As Derek told us, “marketers are always looking for case studies and an advocate for a referral from their current customer base. The common practice so far was to send an Email to anyone and see who will give you a case study.”

And here comes the power of scoring your product engagement; according to Derek, “now you can actually say, well let’s just take the top 10% most engaged target customers and just talk to them. Let’s not embarrass ourselves by talking to people that aren’t engaged with the product and ask you for a referral.”

A very interesting part of the discussion we had on the applications of the Scoring Methodology to marketing, was the fact that scoring through product engagement can empower lookalike campaigns. According to Derek, “with today’s current marketing channels and technology, lookalike campaigns are obviously a big part of the marketing execution.”

And he continued in the importance of product engagement saying that “the better source for your lookalike campaigns to be optimised for your target audience are your best users, your most engaged users. Thus, if you take a look at your most engaged (on a product level) accounts or users, you can understand who is better suited to be highly retained. Except for acquisition, this can help you understand how to attract long term target customers for your SaaS.”

Wrapping This Up

Throughout the interview with Derek, we identified something fundamental: That getting closer to your customers is one of the best ways to keep them engaged with your product and can give you a competitive advantage.

As Derek told us just before we close the interview, “I try to spend at least three to four hours a week, chatting with customers or responding to Intercom chats. And, generally, I’m up late, and if a chat comes in late, generally I’ll pick it up if the team isn’t there. To be honest, it’s some of my favorite work hours, when I’m talking to customers through a chat.”

It’s something that we’ve repeatedly mentioned here at Growth Sandwich. It’s critical to get as close to your customers as possible. According to Derek, it’s incalculable how being close to your target customers helps.”

The Scoring Methodology is one of the best ways to leave the guesswork out of your sales, marketing efforts, and customer support operations. Now that you know how it works, all you need to do is start using it for your own business. To learn more, visit Sherclock’s website and sign up for a free trial.

About the Author: Aggelos M

Aggelos M
I have over 8 years experience in Digital and Growth marketing. Currently, running Growth Sandwich, a London based Product Growth Lab. Before turning in Growth Marketing and Product Growth, I had the chance to pass by head marketing and head digital positions in Athens and London, work with numerous tech startups but also build and run companies. In my spare time, I am consulting ambitious startups about their Growth and Business strategy.