Leave a comment
Get the GH Bookmarklet

AMAs

Casey Winters leads the acquisition product team at Pinterest, which focuses on increasing traffic to Pinterest and converting people into engaged Pinners. Casey has worked on various elements of the growth stack at Pinterest over the last couple of years, amidst massive growth and a shift toward an international first mindset. Even with this international mindset, he still can't understand French after decades of trying. Pinterest crossed 100 million monthly actives users in September 2015.

Prior to Pinterest, Casey was the first marketer at Grubhub, and built out most of the marketing channels there, including paid and organic search, offline marketing, email marketing, loyalty programs, and conversion optimization. During this time, Grubhub spread from three cities to over 500, and went public in April 2014. Casey is responsible for some of Grubhub's worst performing ads, but definitely not its best.

Prior to Grubhub, Casey worked for a few years at Classified Ventures on the Apartments.com, Homefinder.com, and RentalHomesPlus brands, working in various online marketing roles. He has a BBA in Marketing from Loyola University Chicago and an MBA from Chicago Booth, but he leaves that last part out when he talks to engineers. He blogs somewhat frequently at caseyaccidental.com, but still gets embarrassed when people read it.

You can follow him on Twitter: @onecaseman

He will be live on June 9 starting at 930 AM PT for one and a half hours during which he will answer as many questions as possible.

  • SA

    Shaker A

    10 months ago #

    Hey Casey,

    Thanks for taking the time to be here!

    1)I've heard the argument that you shouldn't really focus on quantitative data (ie analytic tools) in the early stages because you don't have large sample sizes to analyze. What you should be doing is focusing on is the qualitative stuff (user interviews etc). What's your thought on this? When do you think a startup should start focusing on quantitative data? How do you balance doing quantitative research VS qualitative research with limited resources?

    2) You talk about the need to learn sql, when it comes to analyzing data. My question is how do you figure out what attributes would be useful for you to track in your databases, so that you can later query the information using sql? Especially when your just starting up a product with one person working on it?

    3)To become a data driven marketer is it enough to know the basics for statistics and sql or do you need to learn something like the R language? If someone wants to learn how to 'understand and analyze' data, what would you recommend they learn?

    4)There are a million things you could be working on growth at any one time. Can you talk about your process for figuring out
    the MOST important thing you have work on RIGHT for growth? How do you make and prioritize your growth road map?

    Thanks for doing this!

    • CW

      Casey Winters

      10 months ago #

      3)Re: becoming a data driven marketer/languages/learning how to understand/analyze data

      You don’t need to learn R. Basic Stats is fine. R can't hurt, but I don't see many people use it. One of our analysts just started using it, and our data scientists love it. SQL to be data independent. Excel to be analysis independent. A curious mindset is very important here as well. When hiring analysts I’ve often found impact is inversely correlated to years of experience as experienced analysts can lose the curious mindset, and were trained to just run cookie cutter reports all day. Asking the right questions is a bigger component of this than most people talk about, and some people are innately better at it than others.

      3 Share
      • AA

        Anuj Adhiya

        10 months ago #

        Cannot agree enough on the curious mindset piece. Everything else can be commoditized, but curiosity, creativity and passion cannot.

    • CW

      Casey Winters

      10 months ago #

      4) Re: figuring outthe MOST important thing you have work on RIGHT for growth?

      Early on in growth, you pick the part of the funnel that has the biggest opportunity (acquisition, activation, retention, referral, monetization) and focus on that entirely for a time. With bigger teams, you have sub-teams dedicated to each of these areas. In those sub-teams, we’ve found the best way to prioritize is to think about the biggest areas and have focused brainstorms around those areas e.g. mobile web to app handoff. Then, we prioritize based on level of effort, level of impact, and chance of success. I like to separate the last two from each other to make it clear which are more “hail mary" style projects that pay off big if they work, but are unlikely to.

      How do you estimate impact? We use base rate (how many people will experience it if we do it), and to estimate effect on them we look at previous experiment data to get a sense of if it’s like to be 5% change or 50%. We also look at if any other companies have tried it, and see if we can talk to them.

      3 Share
      • CW

        Casey Winters

        10 months ago #

        Replying to Hila: Great to hear about growth process set up in Pinterest. In each of the sub teams, how many focus areas do you usually have at any given time? Only one (i.e. mobile web to app handoff), or could be a few? How long does the team usually keep focused on one area before moving to next? How do you decide when to move to next area?

        It depends on team size. Average is probably 3. We usually have engineers focus on one area and segment that way instead of doing it serially. We set up OKRs at the beginning of the half, and guide our progress on those.

      • HQ

        Hila Qu

        10 months ago #

        Great to hear about growth process set up in Pinterest. In each of the sub teams, how many focus areas do you usually have at any given time? Only one (i.e. mobile web to app handoff), or could be a few? How long does the team usually keep focused on one area before moving to next? How do you decide when to move to next area?

    • CW

      Casey Winters

      10 months ago #

      1) Re: "shouldn't really focus on quantitative data (ie analytic tools) in the early stages because you don't have large sample sizes to analyze".

      Well, the reason people give this advice is because that early you don’t have ANY data. When searching for product-market fit, it’s all about understanding your user’s needs. I don’t think you shouldn’t have any analytics, but I’ve seen a lot of startups messing about with the perfect Mixpanel/Segment/Mode/whatever integration when they just need to talk to a few customers to get real insight. I do believe when it comes to measuring product-market fit, you should be using quantitative tools like cohort analysis, and I’ve blogged about this here: http://caseyaccidental.com/product-market-fit-arbitrage

      When things get bigger, you use quantitative data to understand what is happening, and use qualitative data to understand why. For example, let’s say you try removing passwords from your iOS app, and in your experiment, signups decrease by 2%. You do qualitative research to see it’s people struggling with Touch ID mechanics.

      2 Share
      • CW

        Casey Winters

        10 months ago #

        2) Re: how do you figure out what attributes would be useful for you to track in your databases etc

        As much as possible, you want to set up an environment where you don’t have to specifically think of an event to measure it. So even if it’s one table with every raw event happening (like Pinterest when I joined), that’s better than not logging something. You don’t want to get into analysis paralysis in the early stages, so think smart about what you predict being important, and if something else becomes important and is not logged, go back and do it as soon as possible. One reason I liked Mixpanel when it came was as long as an event is logged, you can build a real time funnel for it. You didn’t need to have thought of that funnel in advance.

        Even now at Pinterest, when we kick off a project, we are hypothesis driven. What are we trying to accomplish? What data will tell us if we are successful? Are we measuring that? If not, how can we? That approach is applicable to any stage of company.

        2 Share
  • AA

    Aldin A

    10 months ago #

    Hi Casey,

    Thanks for being here. Huge fan of your work! Like you, I'm also a marketplace geek!

    1) What, in your opinion, are the key metrics for an online product driven marketplace business, such as etsy or spring?

    2) I know you've said that pinterest builds there own tools at this stage, but I was wondering what off-the-shelf analytic tools you would recommend for startups that are trying to build product marketplaces? And can you also explain your rational for each?

    3)I've been thinking about referrals and how they work in marketplaces. In your TWIST presentation you mentioned that other than actual currency the only thing you can offer for referrals is virtual currency ie dropbox (more space). How do you think about marketplaces and referrals? As someone who's just starting, cash isn't something I have an abundance of. And the way I see it I can't really do discounts either like normal ecom sites who buy there products and have a large margin which can absorb giving 10% off to the referrer and referee VS marketplaces which are supposed have a small transaction rate (from what I understand) to stay competitive (which gives them less margin to play with). Any advice on how how to tackle this?

    4)How do you go about figuring out the optimal transaction rate for a marketplace? Start too low, and if you try to increase it everybody will be up in arms. Start too high and you can't attract sellers. Any advice on how to figure out your take rate and strike the right balance?

    5)Suppose by adjusting the take rate you can attract sellers(ie Seller A pays 10% per transaction VS Seller B which will only come on the platform if you charge 7%) Do you lower the take rate for seller B or refuse, and seller B walk? Do you think there can be variance in the take rate between the sellers or should they be charged the same?

    Thanks again for doing this!

    • CW

      Casey Winters

      10 months ago #

      1) What, in your opinion, are the key metrics for an online product driven marketplace business, such as etsy or spring?

      I’m not very familiar with Spring, but for Etsy most people would say GMV as the top metric. I disagree. I think it’s net revenue. The reason why is that GMV includes just # of orders * order value. Net revenue includes those but also includes commission rate, which is important to building a sustainable business. When Grubhub merged with Seamless, they were optimizing in GMV, and we were optimizing off of revenue. That meant we had built a sophisticated restaurant search result order based on commission rate, and they had an alphabetical sort that had restaurants hacking their name to show up at the top like ~Subway. Needless to say, it was a big opportunity Seamless had missed.

      Other than revenue, we cared about second order rate a lot at Grubhub. If you ordered a second time, it typically created a habit. Depending on the marketplace, there’s an ideal frequency of purchase. You want to track to see if the new user is on that path. On acquisition, volume of new users, CPA, and LTV by channel are key to make sure your channels are profitable. Return rates typically is a key metric for products as well.

      3 Share
    • CW

      Casey Winters

      10 months ago #

      2) I know you've said that pinterest builds there own tools at this stage, but I was wondering what off-the-shelf analytic tools you would recommend for startups that are trying to build product marketplaces? And can you also explain your rational for each?

      I’m afraid I’m a bit out of touch with what tools are on the market since they typically don’t scale to Pinterest’s size. I’m comfortable with Mixpanel, and the startups I advise typically use that. It’s fast, has easy funneling, and it’s easy to understand the event tracking. I’ve heard good things about others like Mode/Amplitude and Segment as an aggregator, but haven’t seen them in practice. A lot of startups like Looker, but to me, learning SQL is not that hard.

      • CW

        Casey Winters

        10 months ago #

        By the way, some people struggle to getting over the hump with learning SQL. If that's the case, just use a tutor or online classes. Wyzant has a bunch of local tutors for it, and there are courses online as well with multiple services. I learned by reading online and bothering our director of technology at the time a bunch.

    • CW

      Casey Winters

      10 months ago #

      3) Re: referrals and how they work in marketplaces.

      Marketplaces generally need to squeeze out enough margin to offer a referral fee. Here’s how it works. Let’s say you want your payback period on acquisition costs to be three months (which is what I recommend for early stage startups). In that three months, the average user spends $100 on the platform, and at a 10% commission you make $10. What you should feel comfortable doing is paying $5 to the referrer and $5 to the referee if an order takes place. Now, you have to see if that $5 is enticing to both sides. It may be too small to work, which means you need to get order size up or commission rate up for it to work (or take more risk with payback period). Generally, this strategy works when marketplaces have frequency that can sufficiently incent the first purchase and make profits on the future purchases.

      You have to watch out for fraud here as people will try to hack this (checking credit card, address, name, etc. helps a lot), and you have to check that the LTV of referral customers does not drop compared to other channels as it could with a discount on first usage. This was never a big channel in the Grubhub days because we found a) people didn’t want to talk about staying home and ordering food and b) people acquired vis discounts had lower lifetime values.

      3 Share
      • CW

        Casey Winters

        7 months ago #

        In reference to Hila: The short answers is you usually don't retain them and have to re-acquire them when they have a need again. The longer answer is to have some sort of way to keep the relationship going in between transactions e.g. blogs/emails about home hacks, saving money, ways to decorate, etc.

      • HQ

        Hila Qu

        9 months ago #

        Hi Casey, if the marketplace has very low transaction frequency (like once every few years), but large single transaction value, for example, a home buying platform, how do you go about acquire and retain your users?

      • HQ

        Hila Qu

        7 months ago #

        Ha, thank you for coming back and reply to my question: )

    • CW

      Casey Winters

      10 months ago #

      4)How do you go about figuring out the optimal transaction rate for a marketplace?

      Well, the ideal way to do is to make the commission an auction. Then it becomes a real market that optimizes around willingness to pay. Grubhub, Adwords and many other businesses do this. At Grubhub, restaurants chose how much of a percentage (or flat fee) they were willing to pay per order, and we would optimize the restaurant search results order based on that. Adwords uses a quality score * CPC, which is fancier, but the same idea.

      If you can’t do that for some reason, you need to understand your customer’s margins and how much of that margin they are comfortable passing along to get additional volume. I'm generally more of a fan of starting higher if you know you'll provide the value. You need the supply side to value your service if it works. That said, I know of marketplaces that charged nothing at the beginning because they needed to prove value in a very competitive market, so you can't be dogmatic about it.

      2 Share
    • CW

      Casey Winters

      10 months ago #

      5)Suppose by adjusting the take rate you can attract sellers(ie Seller A pays 10% per transaction VS Seller B which will only come on the platform if you charge 7%) Do you lower the take rate for seller B or refuse, and seller B walk? Do you think there can be variance in the take rate between the sellers or should they be charged the same?

      Obviously based on the above example Grubhub and Adwords have this. So I think that is ideal to have variance. If you can’t do that, it’s a question of whether you have a supply side problem or not. If you need the supply, then they set the price. Hopefully, you can tip the power dynamics over time with how well you do on demand that you can get a fair rate.

      At Apartments.com, we renegotiated rates upward every year with apartment rental companies, and it was neither our sales team's nor our customers' favorite time. But every year we were able to increase our rates because we showed more and more value.

  • AW

    Abdelrahman Wahba

    10 months ago #

    Hey Casey,

    I wanted to ask you, what is the aha-moment of Pintrest?

    Thanks,
    Abdo

    • CW

      Casey Winters

      10 months ago #

      The aha moment of Pinterest is when you see specific, relevant content to you you didn’t know existed. So, our entire signup flow is about understanding what content you will like and getting you to a feed of that type of content as soon as possible. It’s something we’re always working on improving.

      6 Share
      • AW

        Abdelrahman Wahba

        10 months ago #

        Thanks for your answer.

        Follow-up questions:

        1. what's the time-window, where you need your user to reach the aha-moment or else you risk losing them?

        2. What was the path to discovering the users' aha moment and then acting on it?

        Thanks again :D

      • CW

        Casey Winters

        10 months ago #

        There are two ways to think about the first question. One was is to figure out where you cohorts flatten. That's at about a month for Pinterest. The other way to think of it is it has to happen on first session. We bias toward the latter, but track toward the former as day 0 engagement can be misleading.

      • CW

        Casey Winters

        10 months ago #

        For your second question, don't know that I have an answer beyond a shit ton of testing and user research.

  • TS

    Terence Strong

    10 months ago #

    Hi Casey:

    Thanks for doing this!

    I am trying to build out a referral program for a subscription box. My customers share things in person and do not really share online. What tips would you give me? And/Or what things would you try?

    -Terence

    • CW

      Casey Winters

      10 months ago #

      I’d probably need to learn a little more about your customers and the product, but when services are experienced/shared offline, physical evidence can be a powerful tactic. At GrubHub, for example, we put stickers on all the restaurant windows we worked with, physical promotional cards, etc. Lyft is perhaps the greatest example with the pink mustache. I’ve written about this more here: http://caseyaccidental.com/the-power-of-physical-evidence/

      At Grubhub, we realized WOM was low because no one wanted to talk about staying home at Friday night and ordering pizza while they should have been out partying. So we decided to make our brand really funky and playful so that if people wouldn’t talk about using us, perhaps they’d talk about our crazy ads.

  • RB

    Ry B

    10 months ago #

    Hi Casey,

    Awesome having you here! I saw your presentation on retention recently, great stuff.

    -What was your process for identifying the restaurants you wanted on the platform, to convincing them to giving you a shot? How did the acquisition process work at scale?

    -What advice do you have when it comes to building community in a marketplace for the buyer side and seller side respectively?

    -You talk about great customer service being key for grubhub. How does customer service differ for a marketplace? How closely does the restaurant and grub hub work on customer service issues?

    Ex. If someone had a bad experience on the platform and Customer Service at grub hub decided to give them their money back would that be a decision in conjunction with the restaurant or is that Grub hub's call? Also in that scenario where the customer is refunded the money does grub hub alone absorb the cost or is it absorbed in conjunction with the restaurant?

    -In your presentation about retention you talk about choosing the correct intervals for cohorts. How do you decide what is the correct length of time for the cohort & what is a healthy number of visits for the buyer to have, to your site during said time frame?

    -Does understanding and increasing retention work the same for sellers and buyers in marketplaces, or are there differences? If so, how does seller retention differ?

    -In your presentation you talked about removing frictions. Ex. getting restaurants to drop the min order fee. Other than showing them the proof that it's better for them, what else did you do to get them to agree? Did you incentivize them to drop it by using any market mechanisms (ex. better pricing with grubhub if they dropped it?) --- How did people react to not having a min order fee? Did they spend more? Less? Did they abuse the fact that there was not min order fee and order very little (thereby making it unprofitable for restaurants to deliver)

    -Speaking of frictions, I Imagine delivery charges would be a big friction too. Did you get restaurants to do free delivery? If so how did you convince them to do it.

    Thanks,
    Ry

    • CW

      Casey Winters

      10 months ago #

      What was your process for identifying the restaurants you wanted on the platform, to convincing them to giving you a shot? How did the acquisition process work at scale?

      We mainly used Yelp reviews to start as well as local knowledge. Then, we optimized for neighborhoods and cuisines where we had demand. We knew if we got a certain # of restaurants in a cuisine/geo combo, we could have a conversion rate from paid search that would give us a CPA under the revenue we'd make in six months.

      For convincing restaurants to give us a shot, you have to realize restaurants have been sold every scam under the sun. So they are very skeptical. We focused on removing all friction and making the process no risk and very easy to understand. So, you could cancel anytime. You only paid when you got orders. You got orders via a piece of technology you already used (the fax machine, which was ubiquitous in restaurants). There were no hidden fees. And most of the time, you didn’t really pay us. We just subtracted our fee from the credit card amount we sent you, and we direct deposited cash weekly into the restaurant's bank account.

      The restaurant side was all sales driven. At first, we had local salespeople, then switched to an inside sales model over time. Some markets still needed a face to face meeting. Most didn’t.

    • CW

      Casey Winters

      10 months ago #

      Re: How does customer service differ for a marketplace? How closely did the restaurant and grub hub work on customer service issues?

      At Grubhub, we decided early on to accept ownership over the order. If the person’s food was late, that wasn’t our fault, but the user would associate the experience with Grubhub, so we took responsibility. What we did was just make sure to take of the user no matter what in the early days. So we would comp the order, give a discount, whatever, no matter what, but try to get the restaurant to help us. We would absorb the cost if we needed to, but try to get the restaurant to do so if it was their fault. Restaurants are used to this type of customer service, so they generally want to take care of the user as well.

      So, process-wise, when they call you, which we ended up recommending, we said we'd get back to them and take care of it, talk with the restaurant, then decide if the restaurant would comp or we would.

      3 Share
    • CW

      Casey Winters

      10 months ago #

      Does understanding and increasing retention work the same for sellers and buyers in marketplaces, or are there differences? If so, how does seller retention differ?

      A lot of the same principles, but sellers are usually more motivated. By default, they are on the platform more, so usually it’s easier to tell if they are at risk based on how well they are doing on the platform. Are they receiving enough orders compared to their peers? Have they started rejecting orders? They are also usually on or off. So you know if they churned. With consumers, you don't really know if they're never coming back or just busy.

      The other thing to consider with seller retention depending on the marketplace is what Grubhub called “involuntary churn,” which means the restaurant went out of business. This can be a high number with marketplaces that work with SMBs as the failure rate is very high for SMBs.

      3 Share
    • CW

      Casey Winters

      10 months ago #

      What advice do you have when it comes to building community in a marketplace for the buyer side and seller side respectively?

      I’ve actually never seen it happen at scale. I think it’s tough for a business to be both a community and a place where people transact. People tend to think of you as one or the other. Yelp is struggling with this, Houzz is, etc. We're still figuring a lot of things out with Buyable Pins.

      Andrew Chen has a good quote that you don’t just design your site, you design your community too, and that’s why Pinterest is super friendly, and YouTube comments are the most vile vitriol you’ve ever seen. So communities are about creating norms for the service, having people that uphold and enforce those norms (great users, moderators, community managers), and figuring out how to make that scale.

      Fun fact: I was a moderator on the IGN boards in the early 2000s. Had to ban quite a few people.

      2 Share
    • CW

      Casey Winters

      10 months ago #

      In your presentation about retention you talk about choosing the correct intervals for cohorts. How do you decide what is the correct length of time for the cohort & what is a healthy number of visits for the buyer to have, to your site during said time frame?

      When building a product, you have to decide or understand based on the market what type of frequency makes sense. For Apartments.com, people aren’t going to search for apartments once a month no matter what we did. Then you set a goal, and if you start hitting it, you make the goal harder. For example, Facebook always goaled on monthly active users, but as the service got more engaging, a monthly active user didn’t indicate a lot of engagement compared to a normal user, and almost the entirety of the internet population was already an MAU. So they switched to daily active user.

      At Pinterest, we similarly started with monthly, then switched not only to weekly, but an activity metric during the week (saving). Perhaps at some point, we’ll switch to a daily target.

    • CW

      Casey Winters

      10 months ago #

      In your presentation you talked about removing frictions. Ex. getting restaurants to drop the min order fee. Other than showing them the proof that it's better for them, what else did you do to get them to agree? Did you incentivize them to drop it by using any market mechanisms (ex. better pricing with grubhub if they dropped it?) -

      In the early days, we did promise more promotion in the product and even local press. We did not lower their prices. Then we worked on case studies showing the impact.

    • CW

      Casey Winters

      10 months ago #

      How did people react to not having a min order fee? Did they spend more? Less? Did they abuse the fact that there was not min order fee and order very little (thereby making it unprofitable for restaurants to deliver)

      We didn’t remove order minimums, but we did agree to a lower one e.g. $10-15. People ordered more often, but with a slightly smaller average order size when they ordered. It was actually more profitable for restaurants because the increase volume was higher than the lower order size.

    • CW

      Casey Winters

      10 months ago #

      Speaking of frictions, I Imagine delivery charges would be a big friction too. Did you get restaurants to do free delivery? If so how did you convince them to do it.

      We never had a lot of success here. This was usually set at the market level. No restaurants in New York had minimums or fees. Virtually every other city had an average of a couple dollars. The reason this was harder to move is that restaurants had built this deep into their cost structure if they had it. That means it would be part of driver's salary or cover gas or car maintenance.

  • ES

    Edward Stephens

    10 months ago #

    Casey - so awesome to have you on here. Definitely some questions i'd love to ask!

    1) What growth challenges have you faced with Pinterest and how can you/could you change the perception that the typical user is female? Or is it even worth tackling this issue and actually it should double down on this strategy?

    2) Really interested by your focus on international growth - it seems like a lot more US companies are seeing the value associated with expansion into the UK and through Europe at the moment. What are key territories for you that show significant international promise?

    3) How prepared did you feel for the move from Grubhub to Pinterest and what skills did you have to drastically work on to meet the new challenges?

    4) Having been responsible for user growth, how do you see companies solving for retention?

    • CW

      Casey Winters

      10 months ago #

      4) Having been responsible for user growth, how do you see companies solving for retention?

      By far the biggest thing is making the core product better, not adding new features, just making what you have more reliable and a little bit better every day. Most companies get too sidetracked building totally new things because that's fun, but that's not usually what matters unless your core product has plateaued (like Snapchat before Stories).

      Other than that, emails and notifications are the easiest win to help retention. They don't solve core retention problems, but they do add more retention on top if they are done well and remind people about the value of the product. At Grubhub, we eventually invested in loyalty programs, and that was very impactful and very fun work.

    • CW

      Casey Winters

      10 months ago #

      1) What growth challenges have you faced with Pinterest and how can you/could you change the perception that the typical user is female? Or is it even worth tackling this issue and actually it should double down on this strategy?

      We face many growth challenges at Pinterest. We are not a social network and provide personal utility to people, which makes us not a viral service. That’s how most of the fastest growing startups have grown (Snapchat, Facebook, Twitter, Uber). We’re trying to grow really fast without that, and it means we need to be more deliberate about growth and smarter about it.

      It’s also a product you need to start using to see the true value of. Our mission is to help people discover things they are interested in, and help them do those things in real life. No wants to hear about a service that makes their life better. People are like “heard that before”. But once people use it they find a lot of interesting ideas they didn’t know existed. We hear stories every day from Pinners about how we helped them get healthier, find new passions, make them better parents. So we actually have to sell Pinterest to new people on a much smaller value proposition at the start, which is just great content in your interests.

      • CW

        Casey Winters

        10 months ago #

        2) Really interested by your focus on international growth - it seems like a lot more US companies are seeing the value associated with expansion into the UK and through Europe at the moment. What are key territories for you that show significant international promise?

        We're having a lot of success in Europe. We're already heavily penetrated in most English peaking countries (UK, Canada, Australia, etc.). We've done really well in Germany and France, which are the two new European countries we've been focusing on. Outside out Europe, we've focused a lot of Brazil which is really exceeding our expectations. We've also done well in Mexico, and are now starting to explore more Spanish language countries as a result.

    • CW

      Casey Winters

      10 months ago #

      3) How prepared did you feel for the move from Grubhub to Pinterest and what skills did you have to drastically work on to meet the new challenges?

      Pinterest is a much more complicated product, and it wasn't clear what to optimize for at the start. At GrubHub, you either make a dollar or you don't. At Pinterest, I was confused about whether we optimize for just visitation, or saving, or adding original content. It took a long time for me (and the company) to wrap our heads around.

      Grubhub grounded into me a lot of the skills needed to succeed here: to talk to engineers using data, to think about scale, and to wear a lot of hats. What was different was company building approach. Grubhub was very lean, raising a $1 million series A and a $2 million series B. Pinterest had already raised half a billion dollars and as such had way more people, but was still just as young a company.

      The growing pains of people without maturity was new for me.

    • AA

      Anuj Adhiya

      10 months ago #

      @onecaseman Re: international growth - how do you decide which geography/ies you are going to go after next?
      Do you find that the playbook is replicable across geographies? If not, which ones have been exceptions to the rule and why?

      • CW

        Casey Winters

        10 months ago #

        Japan is always the exception ;) It's been an island with a distinct culture for so long. The Japanese expect explanations for everything. Go into any store, and you'll see them. So a platform you can use for any interest with not text on it, just images, has had its challenges.

        The playbook is mostly replicable, but with local tweaks. For example, localizing to British English is a pretty unique issue.

        We look at the largest advertising markets primarily, as well as where we have a groundswell of good, local content to provide a meaningful discovery experience.

  • SE

    Sean Ellis

    10 months ago #

    Hi Casey, thanks so much for doing an AMA with us. How many growth related experiments would you estimate that the Pinterest team launches in a typical week? By growth experiment I mean anything in the AARRR funnel.

    What do you think prevents most companies from running more growth experiments?

    • CW

      Casey Winters

      10 months ago #

      We have over 50 running at the moments, but new launches in a week are probably a dozen or so. As we've grown more sophisticated, this has actually dropped a little as some of the low hanging fruit is gone, and we're investing in our own infrastructure to scale, or making more sophisticated (read:complicated) bets.

      What I've seen really slow down teams from running more experiments is getting attached to ideas and polishing them a bunch before running the experiment.

      This can happen at Pinterest too. Our conversions team built a process to get past this by saying iterating quickly was one of its core principles. So, the process there is that any experiment can be shipped by an engineer/designer pair to up to 25% of users, and they on purpose push out MVPs to validate ideas. Once the idea is validated with experiment data, they polish it and ship or iterate. This helped us move a lot faster.

      3 Share
  • MM

    martín medina

    10 months ago #

    Casey,

    Thanks for coming on here and answering our questions today, I have been following your work for quite some time and especially liked your panel discussion at the GHCONF2016.

    You've worked in a variety of different companies and industries over the years doing marketing and growth, how do you transfer over some of the lessons learned to your work today? Also, how do you go about identifying and building out marketing channels when first approaching a growth project especially one in the early stages?

    • CW

      Casey Winters

      10 months ago #

      One thing I’ve always focused in my career is understanding how something works, and then trying to genericize it. Is this a GrubHub-specific insight, or does this work in many cases? Through that, I developed a framework about growing startup businesses that I am comfortable adapting the components of anywhere. For example, our SEO strategy at Pinterest looks very much like our SEO strategy at Grubhub, but we don’t do any paid acquisition at Pinterest, which I spent a lot of time on at Grubhub.

      For early stage growth, I still use my framework (described here: http://caseyaccidental.com/this-week-in-startups). And I pick from that menu which couple of tests to try. If I have engineers, I tend to focus on the product changes that make sense as they tend to have the highest impact. If engineering is very constrained, I pick from the performance marketing programs that make sense for the business. For example, are people searching for this type of product already? Then use SEO. Does the product generate revenue, and is LTV sufficient? Then try SEM and Facebook ads. Does the product get better if another person joins it, and does the user know that? Try virality. Can you give something of value if someone refers someone else and make your money back on it? Then try referrals. How good is the conversion flow, and are there easy fixes?

      For performance, I carve out small budgets to test a couple of initiatives, then focus on the best performing tests for a while until I hit diminishing returns. Generally, with these tests, you're either close to a CPA target or wildly off. I focus on optimizing the ones that are close. Performance marketing initiatives tend to have an efficient frontier, where if you spend more than this amount, you basically pay more for every user you sign up, without getting many more users in return. So I scale up until I hit that. Usually that requires hiring a dedicated person for it.

  • VG

    Virginie Glaenzer

    10 months ago #

    Can you tell us more about who is on your team?

    • CW

      Casey Winters

      10 months ago #

      Every team at Pinterest is a cross-functional team. The growth team is a cross-functional team of designers, engineers, product managers, and analysts. We have 50 engineers, 8 engineers, 7 PMs, and 5 analysts. Very few had previous growth experience. They all report up to functional heads (head of design, head of product, etc.).

  • VG

    Virginie Glaenzer

    10 months ago #

    Hi Casey, Can you please share how you encourage your team to experiment and think out of the box while accepting the fear of falling and not being discouraged by what's not working?
    Thanks.

    • CW

      Casey Winters

      10 months ago #

      We generally don’t get attached to ideas here, and understand that while strategy and insight is important to success in growth, shots on goal is also extremely important. I’d say that’s grilled into us from an engineering culture. One thing that we focus on not so much success or failure, but learning. An experiment is only a failure if we don’t learn something from it.

      As for encouraging experimentation, we believe ideas are a muscle that needs to be built up over time. Your first ideas are going to be really bad, but the only way to get better is to practice. So, on the Conversions team, we have a weekly meeting where people have to bring and discuss one new idea, and we have a form to fill out that helps them think about the right things during idea generation: how many people will it reach, what previous experiments have shown impact could be, etc.

  • SQ

    Simon Quick

    10 months ago #

    Hi Casey,

    What's been the most successful experiment you've done at Pinterest?

    Cheers,

    - Si

    • CW

      Casey Winters

      10 months ago #

      We generally don’t think about individual experiments, but programs. The first “Gift Wrap” experiment where you land on a Pinterest landing page unauth, and we pull up a translucent black banner stopping you from perusing more content was certainly one of the largest. It drove a dramatic increase in signup conversion rate. It’s now a tactic that’s been adopted at almost every site I see now (Facebook, Linkedin, Houzz). So, #sorrynotsorry for making it a bit harder to use sites without signing up. Seriously though, we have a new version we're testing we think is less annoying I'm very excited about.

      Another hugely successful experiment for us was switching the Pin it button to say “Save” instead of “Pin it” recently.

      Our most successful series of experiments was our work to increase the number of people who land on mobile web that install the app.

      6 Share
      • AA

        Anuj Adhiya

        10 months ago #

        " So, #sorrynotsorry for making it a bit harder to use sites without signing up" - @Hila_Qu does that sound like a familiar conversation we've had? :D

      • AA

        Anuj Adhiya

        10 months ago #

        I'm amazed that a simple change in wording from Pin It to Save would be a huge win! In many ways this feels analogous to the famous red/green button test.

        @onecaseman. Can you talk more about that test? Why did the thought to change the text even come up? And was there an in going assumption whether that would be high impact based on some (internal or external) signal upfront?

      • CW

        Casey Winters

        10 months ago #

        Replying to Anuj:

        We did international research last year in France. Pin it in France means something very naughty, so we saw people laughing in research about it. When we went to Brazil soon after, we saw the Pin it metaphor not working there either. We knew it was the right localization move to make, but didn't expect the impact it had.

  • ZD

    Zoe Di Novi

    10 months ago #

    Hi Casey,

    Would love to hear you talk about how you plan out your roadmap and how often you change it based on results you're seeing?

    How do you decide to focus on smaller optimizations / removing friction vs bigger projects to get the maximum users experiencing the current value quickly (great blog post btw)?

    What bigger projects/ideas are you really excited about?

    Thx,
    Zoe

    • CW

      Casey Winters

      10 months ago #

      How do you decide to focus on smaller optimizations / removing friction vs bigger projects to get the maximum users experiencing the current value quickly (great blog post btw)?

      Well, we don't know in advance what will have the biggest impact. Changing the Pin it button to say Save wasn't very complicated, but had a huge impact. But we generally prioritize smaller projects in between the bigger rocks while people are waiting for experiment data.

      What bigger projects/ideas are you really excited about?

      We've been working a lot on growth about improving the speed of our service, and early returns are showing 25% lift in conversion rate from doing so. that's huge for us, and we're just getting to started.

      I'm also excited about testing a service that allows us to easily test new title tags to see its impact on SEO. Our early manual tests were very promising there.

    • CW

      Casey Winters

      10 months ago #

      Would love to hear you talk about how you plan out your roadmap and how often you change it based on results you're seeing?

      Sure. We do half year planning. And the leads of the sub teams on design, engineering and product meet first to think about the major themes they want to focus on for the half. After that, we do brainstorms on those themes with the broader sub-team of engineers, designers, and analysts. Then, we prioritize the ideas from the brainstorm based on effort, impact, and chance of success (I have a little formula in a spreadsheet (Believe I should upsell Sean's new tool for this here!).

      We then organize into a set OKRs that we present to the CEO, and heads of product, eng, and design. If you do a good job on that, they tell you to get going. If you do a not so good job, they will give you feedback. We make all our key results absolute numbers. Otherwise, people can get in the habit of moving percents that don't matter. What ultimately matters is where the absolute numbers for the company end up.

  • LS

    Logan Stoneman

    10 months ago #

    Casey! What an honor - after watching your talks on growth & marketing, it's great to have you here.

    Your career has had a true north towards growth throughout many different positions. What draws you to this sector of business? When did you know this was something you wanted to pursue?

    • CW

      Casey Winters

      10 months ago #

      Since I graduated undergrad, I wanted to be a VP of Marketing. I just happened to start with a job in marketing at a technology company (Apartments.com) as a marketing analyst. That quickly made me realize all the techniques they were using to grow were totally different from what I learned in school, and that it was a suite of them altogether that made the business successful. As I started working on these tools directly, the traditional marketing career necessitated specializing in one of them, and I knew I didn’t want to do that. I thought if I wanted to run a marketing team someday, I needed to know how all of it worked. That pushed me toward more and more startup-y opportunities (first Homefinder.com pre-launch, then Grubhub post-series A) as those were the only opportunities where you could work on multiple areas simultaneously. As I did that, I got closer and closer to the product, and realized that optimizing the product e.g. SEO, conversion optimization, virality actually had the largest impact in growing online businesses. I still called all of it marketing, but as I met other marketers I realized they were very different. And as I evaluated VP of Marketing opportunities, I found I could have more of a impact in what is now being called “Growth”.

  • HQ

    Hila Qu

    10 months ago #

    Hi Casey,

    Super excited to have you here. The other day I listened to your podcast interview on GrowthEveryWhere, and was really fascinated about how you approach SEO at Pinterest. Surprised that you are a pure marketing major, I thought you come from an engineering background (i guess you blend too well with your engineer team:).

    How is your Pinterest growth team composed? How big % are engineers?

    Do you have to teach yourself coding and hide your MBA in order to lead growth effectively? What are some tips you can share for other non-engineer GrowthPM?

    Other than the band name, is there any other reason you use "accidental" in your blog name? :)

    Thanks!
    Hila

    • CW

      Casey Winters

      10 months ago #

      How is your Pinterest growth team composed? How big % are engineers?

      Answered the team question earlier.

      Do you have to teach yourself coding and hide your MBA in order to lead growth effectively? What are some tips you can share for other non-engineer GrowthPM?

      I write a lot of SQL for data analysis, but I don’t code. PM’s are hired to understand the business first, and the engineering second. So focus on the value you add, not the difference between you and the engineers. Empathy for engineering work and willingness to learn the details are more important that coding skills. PM’s are successful based on a value add. If they needed another engineer, that’s what they’d hire. As a new PM, it’s very much about understanding the value add you can create, and it’s very different per team. When I started on the growth team here at Pinterest, I just started taking notes for meetings first, then gradually found other ways to add value.

      As for the MBA, I generally don’t talk about, but it was immensely helpful in learning how to manage people and work with others.

    • CW

      Casey Winters

      10 months ago #

      Other than the band name, is there any other reason you use "accidental" in your blog name?

      I thought my name alone was boring, and my Twitter handle (onecaseman) didn’t sound like a domain. I was a little inspired by my friend Frank Gruber, who at the time had his blog called SomewhatFrank (it’s now Tech.co). But yes, I also like the band KC Accidental.

      • HQ

        Hila Qu

        10 months ago #

        For a moment, I thought you are being philosophical about we can try hard to grow stuff but there is always accidental aspect :)

  • SR

    shahnur rahman

    10 months ago #

    Hi Casey,

    Being experienced with many marketing challenges in your career, what was that one marketing mistake you made in recent years that might have caused a business loss? How did you recover from it?

    Eagerly waiting to hear from you !

    Best,
    Sayed

    • CW

      Casey Winters

      10 months ago #

      Oh, I’ve made so many! I’d say the biggest was at Grubhub in trusting the early data from experiments and not measuring for the long term/really digging into why something was happening. For example, we tried using Groupon and RetailMeNot to acquire customers. The early returns on these users were great, which surprised me as I expected deal seekers to show lower repeat purchase rates. What ended up happening in the Groupon case is that the early redeemers of Groupons were really great customers and have great LTV. The problem was all the people who waited until the groupon was about to expire to order. They never came back. So, all in all, the return was not good.

      With RetailMeNot, what was happening was people were getting to our order page, noticing a promo code field, then googling “grubhub promo code” and RetailMeNot ranked #1. RetailMeNot wasn’t referring these customers. It was just stealing referrals from direct and making us pay for them.

  • KA

    karim Abd El Kader

    10 months ago #

    Hey Casey,

    What are the best growth hacks and organic acquisition strategies you've either seen or can think of fora home made food marketplace?

    Thanks,
    K.

    • CW

      Casey Winters

      10 months ago #

      Hacks suck ;).

      Focus on a growth process and not a list of hacks. Are people searching for what you have? Will they tell others about it? Where are your potential customers? How can you target them/reach them? It's not about hacks.

  • BW

    Brand Winnie

    10 months ago #

    Whats up Casey!

    What are 3 testable ideas you could recommend that would benefit everyone reading this AMA?

    • CW

      Casey Winters

      10 months ago #

      I really don't like giving specific ideas because every company is different. What makes growth work is a process of experimenting what makes sense for your business. Some general areas that apply to many people:

      Making the mobile web to app tradeoff better: We did 15 experiments at least making our upsells more enticing, our UX flow better, and we saw major increases in people using our app and becoming engaged users as a result

      Title tag testing for SEO: It's not just about stuffing keywords in there anymore. it's about making people want to click you vs. the competition.

      Badging on Android: Very few companies are adding badges next to their app icons when they have new information that isn't quite push level importance. It drives a ton of repeat usage that is sustained over time. You have to use device specific APIs to make this work, but Samsung has over 50% marketshare, so....

  • AA

    Aldin A

    10 months ago #

    Tanks for you insightful answers.

    What resources (books, podcasts, blogs)do you use to stay abreast and learn about growth?

    • CW

      Casey Winters

      10 months ago #

      I mostly just talk to other people in the field. Haven't found any great sources. I have an active RSS feed, but it's of a lot of single publishers like Stratechery, Bill Barnett, Wait But Why, Brian Balfour, Andrew Chen, Tomasz Tunguz, et al. I read Moz to get an outside perspective on SEO.

      My favorite books that relate to growth are not about growth:
      The Goal by Eliasyahu Goldratt (optimization thinking)
      Thinking, Fast and Slow by Daniel Kahneman (behavioral economics)
      Crossing the Chasm by Geoffrey Moore (growth strategy)
      The Black Swan by Nassim Taleb (understanding real-world statistics and risk)

  • ES

    Edward Stephens

    10 months ago #

    Hi Casey a few more:

    1) What have you learned from your time at Pinterest about how people respond to visualisations eg what content drives the most engagement?

    2) Is there ever room for Pinterest to move into video?

    3) Leveraging a users visual data how highly personalised can you make the advertising pushed to them?

    • CW

      Casey Winters

      10 months ago #

      3) Leveraging a users visual data how highly personalised can you make the advertising pushed to them?

      At Pinterest, we know based on what you save exactly what you're interested in. We know if you're planning a trip, what food you like to cook, what clothes you like to wear, what projects you want to do, etc.. The premise of promoted Pins is that based on this data we can target ads that are just as relevant as the Pins we'd normally recommend to you. We're not perfect yet, as there is a lot we still have to build on the ad targeting side. But early returns are looking great.

      Most advertising based sources are trading off engagement with interrupting that engagement to show an ad that pays the bills. Pinterest fundamentally does not have that problem. People are on Pinterest to find ideas on what to make, do, buy, and advertising is just another source for those ideas.

    • CW

      Casey Winters

      10 months ago #

      2) Is there ever room for Pinterest to move into video?

      Absolutely. We're a catalog of ideas, and that idea can be expressed as a video. You can already Pin videos, but we should make it easier to add video content in the future.

    • CW

      Casey Winters

      10 months ago #

      1) What have you learned from your time at Pinterest about how people respond to visualisations eg what content drives the most engagement?

      People are visual creatures, and images are the quickest way to scan through ideas. It's very hard to say what drives the most engagement as certain categories work better (due to demographics), certain sizes (due to how our grid works), etc. A lot of bias built in.

  • RP

    Ravindra Paradhi

    10 months ago #

    The answer to the question above will be very useful thing for me to know as well
    Thanks

  • AA

    Anuj Adhiya

    10 months ago #

    Hey Casey - so stoked to have you on!

    What is the toughest thing about growth to you? And has what's been toughest changed - either on a personal level and/or from the perspective of the stage the company?

    • CW

      Casey Winters

      10 months ago #

      Getting buy in from the rest of the company on what needs to be done is usually the hardest for any growth team. It takes a lot of work to build trust that what you are doing is right for the business and not just sacrificing short term for long term, and you have to do with almost every new person that joins not in growth, because it's not well known.

      Product teams are trained to build beautiful products that are technically interesting and to expect the users to show up, and that's just not what happens.

      The other thing that's new and tough is the consolidation of platforms. Going and Facebook owned the web and took over mobile as well, limiting the channels that can drive scalable growth to basically what they own. Growth people used to think of tons of different channels to optimize, and it's really consolidated if you want to drive big numbers.

      3 Share
  • ZT

    Zetong Teoh

    10 months ago #

    Hey @onecaseman, thanks for doing the AMA! Found so many gems of insights in just this one session alone.

    One last question: How do you differentiate the work between marketing and growth? Does Pinterest have a separate, marketing team? What do they do?

    • CW

      Casey Winters

      10 months ago #

      We have a separate marketing team focused on curating content (mainly for new users who we don't have enough personalization signal yet), partnerships, community management, the communication about new features, and brand campaigns.

  • HQ

    Hila Qu

    10 months ago #

    @onecaseman

    When you are hired as the growth lead, how do you make sure the executive and existing product/marketing team have the right expectation for you?

    • CW

      Casey Winters

      10 months ago #

      Not sure I did a very good job of this ;) We did have a growth for a while at this point, so some of this was understood. But certainly not all of it.

      What I focused on was crafting a very clear strategy on what we were doing and why, how it would work, when we would see results, and what kind of team we needed for it to be successful. I spent a lot of time on the operational strategy ahead of the product strategy as it was more important to understand how we'd work vs. what we'd do.

      If you're starting a new role in growth, reading The First 90 Days is a good guide in general to how to be successful, building a 90 day plan, finding allies, etc. It doesn't change much from growth vs. another position on how to come in and build buy-in and be successful.

      One tip I repeat from Andy Johns over at Wealthfront is that if you're starting a new team and have to focus on one area, it's tempting to focus on the biggest opportunity area. But that might not be the area easiest to move. And you want to move metrics quickly to reinforce why you're there. I've written more about this here: http://caseyaccidental.com/building-respect-product-team

  • AA

    Aldin A

    10 months ago #

    Thanks so much for taking the time to give answers with such depth.

    You mentioned Loyalty programs and that they were impactful for you.

    1)How did the loyalty program work? How did the loyalty program work in conjunction with the restaurants? Did the restaurants contribute anything towards the loyalty(ie the incentives that were provided by the loyalty program, was grubhub totally footing the bill or did restaurants contribute somehow)?

    2)What were the obstacles and intricacies you had to overcome to make it work? What did creating this program teach you about creating loyalty programs in general? In your opinion what's different about loyalty programs in marketplaces vs ones not on marketplaces?

    Thanks

    • CW

      Casey Winters

      10 months ago #

      It did not involve restaurants. We looked into that, but it was too complicated. Grubhub foot the bill.

      I've talked about loyalty in general in these two posts:
      http://caseyaccidental.com/loyalty-marketing-strategies-segments/
      http://caseyaccidental.com/loyalty-marketing-program-building-success/

      For GrubHub, we needed to learn which segment to focus on as outlined in that first post. So we surveyed users to understand their loyalty, and data mined our user base to understand frequency. Once we had an understanding that the biggest opportunity was frequent, non-loyal, we had to understand the use cases that people used delivery, but not Grubhub. Once we talked to a bunch of users, we got some major reasons. Then we surveyed our non-loyal users again to get statistical representation of what biggest reason for ordering delivery, but not using Grubhub was. Was it a product issue, a coverage issue, or just habit? It was just habit.

      We thought we could change habits, but the unit economics of loyalty programs suck for marketplaces. When someone spends $30 with Grubhub, Grubhub doesn’t make $30; it makes $3-5. So you model out what you can spend and how much you can influence. In our case, traditional models didn’t work. 25 orders would get you a free drink. Not great. So we got more creative. We made it a game you played on every three orders with a 25% chance of winning and a variable reward (anywhere from a free drink to free food for a year). You generally want to shift to variable rewards in engagement programs instead of static awards, because people become accustomed to static rewards, so they cease to influence behavior. We found that making it a game also increased perceived value.

      You should get a lawyer if you do this, as there are so many weird gotchas. Like being classified as a sweepstakes, which means you need post bonds for each state you operate in, you can’t AB test, and you need to be able to play without purchase.

      So at GrubHub, our game was classified as a sweepstakes, so we couldn’t AB test. So we could either do it in one state and not another, keep it live forever and trust that it was working, or pulse it, like the McDonald’s monopoly game. We chose the latter.

  • AA

    Aldin A

    10 months ago #

    Hi Casey,

    Thanks for taking the time to answer my questions. It great to get perspective from someone who's been there and done that.

    1) As a marketplace scales, matching the appropriate buyers and sellers becomes critical, just because there's so many of each in the marketplace and not everyone is perfect for everyone else. One way to go about this is to build a recommendation engine to help connect the appropriate parties. Did you build a recommendation engine? If so how far into grubhub's life did you build it? What else did you do to ensure the appropriate parties had a higher chance to connect, other than recommendations?

    2)What other problems did you face as you scaled the platform? And how did you solve them?

    3)Can you talk about fraud on grubhub? How did you identify fraud? How did you deal with it?

    4) I read a great article about search in e-commerce and when to implement it. If you haven't read I think you'd like the article:
    http://firstround.com/review/What-I-Learned-the-Hard-Way-Building-an-E-Commerce-Site/

    In essence it says if your product catalog starts out small, you should delay building search if people don't know exactly what to search for and let them explore. You should instead optimize browsing and build out the system for how to categorize the product on your site. This will help you learn & build a better user experience. When you have categories users can drill down based on what they are looking for. This would allow us to learn more about our customer preferences and how they discover products.

    Q: What's your opinion on building out search early on? Do agree and think there is a lot to learn from delaying building search when you're product catalog is small? How would you go about learning from users is this case? Would you use "visitor recordings" to see what they did and
    figure out why, or would you use some other technique? If you've had a chance to read the article, I would love to hear your thoughts on it.

    Thanks for taking the time.

    • CW

      Casey Winters

      10 months ago #

      1) As a marketplace scales, matching the appropriate buyers and sellers becomes critical, just because there's so many of each in the marketplace and not everyone is perfect for everyone else. One way to go about this is to build a recommendation engine to help connect the appropriate parties. Did you build a recommendation engine? If so how far into grubhub's life did you build it? What else did you do to ensure the appropriate parties had a higher chance to connect, other than recommendations?

      We never built a recommendation engine. We did simple sorts instead like popularity, new, lowest minimums, etc. We played around with the order of results a lot based on revenue per order and conversion rate of the restaurant.

      This depends a lot on the type of marketplace. Restaurants almost always have spare kitchen capacity, which means they can serve one to many on the platform. This is unlike Uber, where a driver can only service one rider at a time. So focused on getting supply in both geographies and cuisines where people were searching, and that was it.

      2)What other problems did you face as you scaled the platform? And how did you solve them?

      Customer service is a huge scaling problem, both from a people management and a tools perspective. We invested in chat, phone system integrations, etc. as we scale. Also, when you start scaling to millions in spend, some of the online marketing tools don't really scale well. So, for Adwords, we switched to using Marin Software. Then, Marin even had trouble scaling without our desired level of sophistication.

      For marketing in general, when you start doing a lot of different types of advertising, there are interaction effects. So we had to work hard on refining our attribution model to support properly attributing the value of higher in the funnel activity like TV, blending play data, survey data, promotional data, and referral data. We used Convertro to help us track it all, but built our own models.

    • CW

      Casey Winters

      10 months ago #

      3)Can you talk about fraud on grubhub? How did you identify fraud? How did you deal with it?

      There are many different types of fraud, and some are unavoidable, like stolen credit cards. The chargebacks are a cost of doing business. That said, you can identify patterns. For things like simple promotional abuse, we'd look at credit card, name, and address similarities. Once we got bigger, we invested in enterprise fraud detection tools, but I forgot the name of the one we committed to.

    • CW

      Casey Winters

      10 months ago #

      Q: What's your opinion on building out search early on? Do agree and think there is a lot to learn from delaying building search when you're product catalog is small? How would you go about learning from users is this case? Would you use "visitor recordings" to see what they did and
      figure out why, or would you use some other technique? If you've had a chance to read the article, I would love to hear your thoughts on it.

      I think article is bit too prescriptive for what depends on the business. Building out a taxonomy is generally good hygiene and great for SEO, but it doesn't necessarily mean don't build out search. At Grubhub, we needed search to make sure the restaurant delivered to you. There was no other good way.

      For Pinterest, we probably didn't need search at the beginning because our original discovery experience was higher in the funnel. We added it to go a step lower in the funnel, where you're narrowing down from browsing to a more specific idea.

      • AA

        Aldin A

        10 months ago #

        Thanks for taking the time to share your experiences with me. Really appreciate it.

  • KA

    karim Abd El Kader

    10 months ago #

    Thanks Casey for your feedback
    So will focus on strategy rather than tactics

Join over 70,000 growth pros from companies like Uber, Pinterest & Twitter

Get Weekly Top Posts
High five! You’re in.
SHARE
67
67