Leave a comment
Get the GH Bookmarklet

AMAs

I’m John Egan. I’m the Engineering Lead for the email & push notification team at Pinterest. At Pinterest we have tens of millions of people use our service every day & I’ve learned a lot about growing a company at scale. Before Pinterest, I was at Shopkick for 3 years where I led the Growth Engineering team. Shopkick is a mobile rewards app for retail stores. In the 3 years I was there we managed to grow the company from 1MM users to 8MM users through invites, geofencing, and push notifications. Soon after I left, the company was bought for over $200MM. Ask Me Anything & follow me on twitter or checkout my blog to see my thoughts on growth.

  • LT

    Luke Thomas

    over 3 years ago #

    Hey John,

    I'm curious about the team structure. Are there multiple teams? If so, how is that broken down? Do these teams consistent of people with a variety of skill-sets (i.e. - engineering, marketing, etc)

    Also, I'd be curious how much time is spent optimizing existing "initiatives" vs. exploring new ones.

    • JE

      John Egan

      over 3 years ago #

      We have several sub-teams within Growth. There is Acquisition, which handles signup pages, app installs, and SEO. There is Retention & Resurrection (my team) which handles emails, push notifications, and dormant user flows. There is an Activation team that owns the first use flow & new user drip campaigns. International team owns driving international growth though country specific tweaks & ensuring we show locally relevant content. Finally we have an Invites team this quarter aimed at trying to see if we can grow invites as an acquisition channel.

      I’d say we spend probably about 80% of our time on existing initiatives, and about 20% on exploring new areas.

      • JE

        John Egan

        over 3 years ago #

        Forgot to add, each sub-team is usually made up of a Product Manager, several engineers and several designers. We have some marketers, but they sit outside of Growth even though they will often work closely with the Groth team.

    • DL

      Dylan La Com

      over 3 years ago #

      Great question @lukethomas

  • ET

    Everette Taylor

    over 3 years ago #

    Thanks for joining us @jwegan, my first question is - what tools are the growth team using at Pinterest right now?

    • JE

      John Egan

      over 3 years ago #

      Most of our tools, especially around experimentation & analytics, we have built out in house. However, there are still a few things we offload to third parties. We are currently using Yozio for mobile install tracking. We use SendGrid for email delivery. And we use Tableau for our growth dashboards.

      One tool we built out recently is similar to Optimizely. It allows us to experiment on different variants of copy & it will auto optimize itself. What is cool is it will optimize on a per language basis so the variant that gets shown to German speakers might be different than the variant that gets shown to Spanish speakers.

  • DL

    Dylan La Com

    over 3 years ago #

    It's great to have you on @jwegan. Can you tell us a little about a growth experiment that you and your team are particularly proud of? Either one that was a big success, or maybe was a big lesson learned.

    • JE

      John Egan

      over 3 years ago #

      @dylan, I’ll talk about one experiment that was both a big success and a big lesson learned. At Shopkick we built out geofenced notifications to alert users when they were near a store that had points. We had huge hopes for this feature & spent a lot of time building, testing & tweaking the feature to ensure it was accurate, and not to annoying. When we finally launched it as an experiment it showed only a 3% lift in store visits, which we found very disappointing. We spent a lot of effort debugging and couldn’t really find anything wrong. Finally, I segmented the data by new users vs. existing and we saw that for new users the lift was over 20% & it was having a big impact on new user retention. One of the big lessons I learned, was if your experiment is trying to increase user activity, oftentimes your core users are already using your product as much as they can. So, you have to see how other segments of users (less engaged, new, dormant, etc) are responding.

      • SE

        Sean Ellis

        over 3 years ago #

        Awesome advice at the end to segment the data when judging a winner - especially new vs returning.

  • WW

    William Wickey

    over 3 years ago #

    I'd love to hear more about the '1d7s' you mention in your LeadGenius Q&A http://blog.leadgenius.com/retention-key-sustainable-growth-qa-john-egan-growth-engineer-pinterest/

    "One particular retention metric I like to use a lot is '1d7s' which is basically the percentage of users that visit your app/site a second time in the seven days following signup."

    What percentage of users typically return for a second time during that first 7 day window on both Pinterest and Shopkick? What kind of adjustments have you made to make that number go up?

    Thanks!

    • JE

      John Egan

      over 3 years ago #

      Whenever you’re working in signups you want to make sure you’re actually retaining users and they are still using your product 1 month, 3 months, out. The problem is when you’re experimenting; you can’t wait around a month just to find out if it is working. What we noticed is that users coming back in their first week is very strongly correlated with their long-term retention.

      I can’t share specific numbers, but I would definitely recommend for aiming at a 1d7/signups ratio of > 50%. Two main ways you can drive this number are 1) better communicate the products value (ex: better first use flow) and 2) have emails/push notifications with interesting content on a regular cadence to help users to start to form a habit of visiting your product (ex: new user email & push drip campaign)

    • AA

      Anuj Adhiya

      over 3 years ago #

      I'd like to add on to this question - is '1d7s' the one metric that matters for Pinterest or is it something else?

      • JE

        John Egan

        over 3 years ago #

        The one metric that matters for Pinterest is WARCs (Weekly Active Repinners or Clickers). However, 1d7s would be the one metric that matters for our signup & activation team.

  • MB

    melissa blume

    over 3 years ago #

    Morning @jwegan,
    How do you temper the number and type of push notifications or emails to send? Have you done any experiments that have informed those decisions? I'm curious if you've seen CTR trends change over time and how you've improved or reacted to those.

    • JE

      John Egan

      over 3 years ago #

      Currently, the way we try an control the number of types of push notifications/emails is by user targeting (i.e. targeting less engaged users, or users who have not received a notification in X days) & basic rate limiting both overall & by type.

      Longer term we have several projects under way to get a lot smarter about this and figure out which types of notifications a particular user responds to and which ones they don't so we can start to optimize our targeting and rate limiting on a per user basis (i.e. personalization)

      As for CTRs, it is not uncommon to see a new email or push notification will initially have a higher CTR before settling down to a baseline. However, once it has settled, it is usually pretty stable over time. The CTR also usually settles pretty quickly (i.e. after a user has received it a couple times).

  • ED

    Earl Dos Santos

    over 3 years ago #

    Hello @jwegan - who are the growth hacking experts outside of Pinterest that you most respect and/or learn from?

    • JE

      John Egan

      over 3 years ago #

      When I was first starting in Growth learned a lot from reading & following Andy Johns, Andrew Chen, Sean Ellis, & Nir Eyal.

      There are many others I respect, but just to name a few more: Hiten Shah (KISSMetrics), Gustaf Alströmer (AirBnB), & Josh Elman (Greylock)

  • MB

    Morgan Brown

    over 3 years ago #

    Hey @jwegan thanks so much for doing this AMA with us. What does a typical day on the Pinterest growth team look like?

    • JE

      John Egan

      over 3 years ago #

      Hey @morgan, I’ll talk about a typical week since that might be more insightful. Usually the week kicks off on Monday with a meeting of all the Growth Leads to discuss any problems/issues/blockers with projects on deck that week. The biz analytics team also meets on Monday where they go over all key metrics and if any have unexpectedly drop, they will assign an owner to investigate. Tuesday we have a sync with the whole growth team + representatives from several other teams (user research, community, etc) where we discuss how much we grew last week, what drove that growth, and a summary of all the projects in flight. Finally, every other week there is a meeting with the CEO & leadership team to discuss how growth is going, if we are on track to our quarterly goals, and high-level growth strategy.

  • AB

    Adam Breckler

    over 3 years ago #

    Hey @jwegan, i'm curious to hear about your process for discovering and communicating insights around growth inside the growth engineering team and across other teams? What does the typical flow look like from posing a question of your data to analyzing it to coming up with a hypothesis and proposed course of action or experiment?

    • JE

      John Egan

      over 3 years ago #

      Each sub-team in Growth is responsible for coming up with their roadmap & figuring out which experiments they want to run. Usually each team has a model for how their piece of Growth works (ex: for my team it is: # of users emailed * # of emails per user * click through rate per email * % of users that repin from a clickthrough). Data analysis can help with figuring out which piece of the formula to focus on, but then it is more intuition & creativity to figure out what experiments you want to run.

      When it comes to running experiments we have a process in place where you write up an experiment doc, which includes the hypothesis, screenshots of the experience, and results. This experiment doc gets sent out to the whole company whenever you launch or end an experiment. This helps everyone be aware of what experiments are running, chime in if they are interested, and see what the results were.

      • PC

        plc clark

        over 3 years ago #

        can you speak a little about how you roadmap? quarterly? 12 months?

        • JE

          John Egan

          over 3 years ago #

          Priorities on Growth can change pretty frequently so we do our roadmap on a quarterly cycle. Even then though, the roadmap will continue to evolve as the quarter goes on.

  • SE

    Sean Ellis

    over 3 years ago #

    How do you decide which growth lever for Pinterest you should be focusing on at any given time (retention vs acquisition vs activation…)? Or are specific people/teams assigned to each lever perpetually? Also, within a given lever how do you prioritize experiment ideas?

    • JE

      John Egan

      over 3 years ago #

      We have a process called MAU Accounting where we track # of new MAUs this week = New signups - churned signups + resurrected users – churned users. This way we can see how much each component of growth (acquisition, activation, retention, resurrection) is contributing to our growth. We have one team assigned to each lever & figuring out which level to focus on is a factor of how much it contributes to our growth * how much we think we can move it. There is usually at least one PM & one engineer assigned to each sub-team on a permanent basis so they can develop expertise in that area, but we re-balance the rest of the engineers & designers on a quarterly basis.

      Within a team we usually try and prioritize ideas based on ROI. We make a swag estimate of total number of MAUs each experiment will contribute & how much effort the experiment will take. We also however usually set aside at least 20% of our time for more speculative bets where estimating the probability of success is difficult.

  • LS

    lynly schambers

    over 3 years ago #

    What tools or services would you recommend for growth hacking if team is not large and budget is limited.

    • JE

      John Egan

      over 3 years ago #

      I've been hearing a lot of great things about http://segment.io lately. It allows you to swap out different analytics providers without changing a line of code. That allows you to find the provider that fits your budget & needs and enables you to switch providers as your budget and needs change.

      • LS

        lynly schambers

        over 3 years ago #

        Thanks so much for the response! Very helpful. I have reams and reams of digital and product marketing experience but this is a new area for me that I am trying to get more involved in as I agree it is critical to test, learn and repeat as success indicates. One other area I am looking into is how growth hacking applies when your product is an enterprise level SaaS platform targeted at C-level buyers - so not a B2C play for acquisition. Any high level thoughts on how growth hacking may differ in approach depending on if you are going after consumer vs enterprise market? Thanks again!

        • JE

          John Egan

          over 3 years ago #

          This is a bit outside my area of expertise by the advice I would give is
          1) Make sure your product isn't a vitamin, make sure it is a pain-killer. http://slidesha.re/1zlXgiW
          2) Figure out the most effective way to reach your target audience (sale people? advertising?, conferences?)
          3) Figure out what value your product provides them & figure out how to quickly and clearly communicate that value
          4) Follow up. If they don't bite at first, follow up a few months later.

  • SK

    Sakari Kyrö

    over 3 years ago #

    Hi,

    Have you at any point consciously tried to get more women to join, or has the gender division been a natural outcome?

    • JE

      John Egan

      over 3 years ago #

      I think the initial gender skew was more of a result of Pinterest initially getting traction within the female demographic & us just running with it. We actually have been working a lot the past year or two to target male users. Some of it is basic stuff like making sure if a guy joins the site, his feed isn't full of wedding dresses of if he searches for "shoes" he doesn't get a page full of high heels. A lot of this work seems to be paying off now, and last quarter, males were one of our fastest growing demographics on Pinterest.

      • SK

        Sakari Kyrö

        over 3 years ago #

        Thank you for the reply! "We actually have been working a lot the past year or two to target male users." << Very interesting!

  • EK

    Eetu Karppanen

    over 3 years ago #

    Hello @jwegan! Would be happy to hear about how big part localization has been playing in growth of Pinterest? I was working before as social media consultant and was guessing alot about how big amount of users you have got from Scandinavia (especially Finland) after you did service localization here.

    • JE

      John Egan

      over 3 years ago #

      There are a few parts to localization
      1) Localizing the app is just a basic step you need to take to grow in non-English speaking countries

      2) Next for us is sourcing local content. In other words, getting Scandinavians to add new Pins they find interesting so we can surface those Pins to other Scandinavians. This helps because descriptions are in the local language, the content is what people in the region find interesting & matches the tastes of the region, etc. To do this we promote the pin-it extension & bookmarklet pretty heavily internationally

      3) Connecting with locally relevant influencers and brands. To do this we have country managers that are responsible for getting locally influential bloggers and brands on the service. This obviously is pretty high-touch so we only have country managers in a handful of countries, but we continue to scale this out as resources allow.

      Each one of these have been a pretty significant driver of growth within a particular country.

  • MB

    Morgan Brown

    over 3 years ago #

    Hey John, another question for you. What are some of the big differences between working on growth at scale vs. early growth and the things you did at Shopkick?

    • JE

      John Egan

      over 3 years ago #

      1) At earlier stage you spend a lot more time trying out lots new things and discovering what works and what doesn’t for your product. Later stage you’ve largely figured out what works and it is all about doubling down on that.
      2) You can get data for experiments a much faster when you have millions of users to test things out on. :)
      3) At a later stage you have to be a lot more conscious about how experiments impact the user experience & the brand since everything you do is under a lot more scrutiny. For example, just asking for a new permission on Android at a late stage company could potentially you negative press whereas at an earlier stage company, nobody will notice.
      4) On thing I found interesting is at a late stage company, growth is much more predictable. Based on historical data & what experiments you have in the pipeline, you can usually guess how much you will grow next week or next quarter pretty accurately.

  • YV

    Yuvrajsinh Vaghela

    over 3 years ago #

    What are your strategies to grow user acquisition, and make them active on Pinterest ?

    • JE

      John Egan

      over 3 years ago #

      When it comes to acquisition, figure out what your strength is and how to exploit it. Pinterest has a lot of content so SEO, social sharing, etc are key acquisition strategies.

      To make them active, it is all about clearly communicating the value your product provides the user. In our New User Flow we have new users tell us their interests to we can generate a feed of content the user will hopefully find interesting & immediately connect with.

      • YV

        Yuvrajsinh Vaghela

        over 3 years ago #

        Might be indefinite, but informative !

        Thank you john for sharing your ideas.

      • YV

        Yuvrajsinh Vaghela

        over 3 years ago #

        Hey, John

        I think, you guys can add, "follow user" or "follow button" directly under the pins.

        Because, I've noticed, It is very hard to get followers, and get engaged with them, for second tier follow button.

        I think, it can engage more users, once they can easily get niche related followers like twitter.

  • AA

    Andrew Allsop

    over 3 years ago #

    What skill stack do you think would be most valuable for someone new to the field to focus on developing?

    I found myself on your GitHub the other day trying to answer this question, but that doesn't answer any that are non-technical.

    • JE

      John Egan

      over 3 years ago #

      Depends a lot on what you want to do (PM? Engineer? Marketer?). One skill I think is useful for anyone working on Growth (engineers, PMs, marketers, etc) is SQL. Sometimes theres questions that analytics tools can't help you with, and it is a valuable skill to be able to dig into the data yourself. @justinmares wrote a book called SQL for Marketers that might be a good place to start (https://www.udemy.com/sql-for-marketers/)

  • MF

    Mateo Folador

    over 3 years ago #

    Hi jhon
    What was shopkick strategy for acquiring the first users
    Thanks

  • ZT

    Zetong Teoh

    over 3 years ago #

    Thanks for doing the AMA, hopefully I'm not too late on this one.
    You are the lead engineer of your team, however, in what context of growth (be it ShopKick or Pinterest) requires your technical skills?

    • JE

      John Egan

      over 3 years ago #

      I believe having engineers that specialize in Growth is essential for any company trying to maximize Growth. For instance the Facebook API is always changing, the functionality of push notifications changes with each release of Android/iOS, etc. It is important to have someone who is both technical and creative who can stay on top of the changing ecosystem and spot new opportunities.

      An example of this is Local Notifications on iOS were a virtually unused feature by most apps on iOS. However, Local Notifications looked and acted like push notifications, except they were not tied to the push notification permission. I realized the significance of this and I was able to effectively use this little known feature for very significant gains until Apple tied local notifications to the push notification permission in iOS8.

  • AD

    Adam Daigiain

    over 3 years ago #

    Hey John, I'm curious to hear about some of the major findings you had at Shopkick with invites / referrals. Also, what kind of metrics did you pay attention to in those flows? How did you follow those tests through the entire funnel. And what would you recommend to someone who's working on something similar :)

    • JE

      John Egan

      over 3 years ago #

      Shopkick had incentivized invites and we found playing with the incentive structure a lot helped. Also if someone is willing to invite people, finding ways to maximize the number of people they invited was also very effective.

      For metrics we just paid a lot of attention to the invite funnel, and the conversion rate throughout the funnel. We found it very helpful to segment the funnel by which prompt the user saw, because we found that say an invite prompt in first use, performed very differently than an invite prompt on the user's profile. One might get a lot more invites sent, but then have a much worse conversion on the receiver side.

      Finally, one thing you always have to be careful of with invites is making sure you're optimizing for getting people on to the service who will actually stick around. So make sure you monitor the retention rate of invited users and it should be pretty close to, if not better than, organic users.

      • AD

        Adam Daigiain

        over 3 years ago #

        Thanks @jwegan for the detailed reply. Currently looking at messaging and friction. Our pm doesn't want to use monetary incentives so that will be a challenge. That and setting up all tracking necessary to monitor the entire flow and watch what happens down the funnel.

  • KY

    Kenzou Yeh

    over 3 years ago #

    Hello John,

    thanks for sharing your experience. btw, can you share one of your biggest challenge either in Pinterest or Shopkick and how you make it? thanks

  • GG

    Gab Goldenberg

    over 3 years ago #

    Do you realize that saying "I left and then they exited for $200M" means that you were holding them back from $200M :D?

  • EL

    Elyse Lefebvre

    over 3 years ago #

    Thanks for all this, John!
    I was wondering if you had any suggestions for growing lower engagement products, where users only come to the site a few times a year. Retention metrics like 1d7s don't really work for us, but 1d90s is a bit tricky..

  • FP

    Francesca Piazza

    over 3 years ago #

    Hi John! I would like to know if becoming a growth hacker was a decision or if you just found yourself in that role, and how did you start. Also what are the steps that you follow within your strategy. Thank you so much! You can reply me on twitter as well, I am @persulla

  • OT

    onur turna

    over 3 years ago #

    Hey John,

    I'm curious about your experience from the early days of Shopkick, where you had few users and few brands in the app. There is a chicken&egg problem actually; in an app with low users, it's hard to list big brands.. and in an app with no brands listed it's hard to attract and retain users. We're working on a similar retail app in Turkey. I'd appreciate if you could share your experience on the early day struggles. Best,
    Onur

  • YV

    Yuvrajsinh Vaghela

    about 3 years ago #

    Hi John,

    I've wrote few points which can drive more engagement for the Pinterest users.

    Hope you would love it.

    https://growthhackers.com/pinterest-awesome-3-features/

Join over 70,000 growth pros from companies like Uber, Pinterest & Twitter

Get Weekly Top Posts
High five! You’re in.
SHARE
46
46