Developer Tooling with Scott Woody
We sat down with Metronome CTO Scott Woody for a live Ask me Anything (AMA) event to talk about his experience with developer tooling, building versus buying, and delivering value to users. You can listen to the AMA or read our recap below and follow us @Seaplane_io for details on upcoming industry events!
Scott is the CTO of Metronome, a startup that helps software companies launch, iterate, and scale their business models with billing infrastructure that works at any size or stage. Previously Scott was the Director of Engineering at Dropbox, where he worked on internal tooling.
Can you tell us a little bit more about yourself?
So I started in Silicon Valley around 2011. I started a company back then focused on building recruiting software, specifically applicant tracking systems, and that product failed to find significant product market fit. So I joined Dropbox in the early part of 2013 through an acquihire. Dropbox was kind of where I learned to be an engineer and got to see a lot of the benefits of engineering at scale. At the time, Dropbox was about 200 people. I then spent about six and a half years at Dropbox doing a bunch of different things like working on internal tools and some UI front end stuff. Then I moved into growth engineering where I took the team from two engineers up to 60, 70 engineers over the course of three years, going through an IPO process along the way.
I would say Dropbox is where I “grew up” as an engineer and got to see what [high functioning teams] really look like. And then after that I decided to start a new company, that’s Metronome, and we've been working on it for about two and a half years. Now the team has roughly 30ish people, about half of whom are engineers. I’ve [seen], you know, startup one, startup two. That’s my super quick history inside Silicon Valley.
You mentioned that Dropbox had a philosophy of building a lot of their tools in house. What was that like and what are some of the benefits and downsides that you that you saw with that strategy?
Yeah, it was super interesting. I think this is like 2013 and a lot of the leadership at Dropbox was kind of, understandably, very proud of our engineering team. They were incredibly good engineers. So what they decided was that for a lot of problems it made sense to insource the solutions. I think at the time the kind of externally available solutions to a lot of these problems were honestly pretty bad. So they were like “well, we could do a better job in a couple months and then let's go and fund this and fund that.”
What it meant was that for a few years there, Dropbox just had a ton of internal investment in these internal developer tools and internal systems. In fact my startup, which built applicant tracking systems, became the applicant tracking system of Dropbox for a number of years.So I think in the short term what it meant was a lot of engineers got to turn in work, got to focus on the people in the company as our customers, got to really invest, and I think that works really well in a time where you have a lot of really talented engineers who are really passionate.
But the downside of that strategy is eventually the market catches up, right? Let's take any given thing that you do internally. First build, let's say some kind of deployment tool, you kind of design it for the company as it exists at that moment in time. It fits like a glove to your internal processes. But then what happens when your engineering team goes from one hundred to 300 people? It's like that system can’t scale with it. And so what ends up happening is that internal system you've built all these dependencies on, it starts to show its age. It starts to creak and crack. Then you start to realize that actually, maybe it wasn't the best idea to like continuously staff these things.
That was the story that I saw playing out across a number of different things. I think the tools that we had at the time were great, but they aged out. I think this is true for almost any [tool]. We had our own internal feature flagging system. We had our own internal experimentation platform.We had our own internal deployment systems. And I think what you just would see is that these systems would start out amazing and then over time they wouldn't get the time and attention that they needed to kind of grow and flourish with the company. I think the company also started to focus its resources more on, you know, the kind of core value that it would provide.
I think in the pro camp, it was really fun. You got to have really good engineers working on these problems really tightly, and you kind of built perfect solutions for that moment in time. But in the con camp, I feel like a lot of these solutions didn't age particularly well. Over time you kind of build these hard dependencies on these internal systems which means that, over time, [if] you wanted to switch to something like CircleCI or something like that, it’s actually really hard because you built tons of internal dependencies on it. So switching costs were super super high. I think in 2013, 2015, the strategy was defensible and maybe even made sense but in 2022, I think the landscape has just changed so much. The tooling has advanced way beyond anything that could have been built internally.
With that in mind, are you still building any developer tools in house at Metronome, or are you picking whatever is available in market and using that instead?
The guidance I've tried to give to the team is like, we have very few innovation points that we're allowed to use. We're a tiny team. So [we want to] make sure that they're focused on things that are going to drive core value to the client. In general, what that leads to is a “buy-focused” philosophy.
There are some problems, however, that maybe we actually do need to invest in and build our own tools. I think that's a possibility. And I think generally, when that happens, it's because we can't find a great solution in market that can solve this. Stripe very famously has a “sandbox mode” for their application that allows them to toggle between [environments]. We couldn't find a good tool to enable that in the market, and we really wanted users to be able to bounce between the sandbox and production environments with no friction. We ended up spending a month or so building out a system that could do that. We want to conserve our innovation points toward the things that are truly differentiated in our product landscape.
A lot of startups pop up from internal teams building internal tools, and saying “hey, this is actually a product.” What’s the impact of people picking tools off the market instead of building them internally on the company ecosystem as a whole?
I think that's a great question. As long your litmus test is “look on the market first” if there's a solution there that’s good, use it. If you go to the market and you say “there is no solution there,” then yeah, it's a reasonable solution to go build it. Then you get this kind of flywheel that you're talking about.
Having the first step be “look at the market” actually strengthens the outcome of the ecosystem, because what it means is that if you look at the market, and you see there are five competitors who are all doing more or less the same thing, and you don't have a unique angle to it, and they all more or less solve your problem — why are you doing that internally? Go put your innovation towards something that's underserved.
I think authentication is a good example of this, where if you look at the Stitch team out of Plaid, they looked at the market, they said “the best thing here is Auth0,” then they said “actually, I think we could do something significantly better than this. I think there's a new angle to the way we think about authorization that we should go and spin out.” So I'm all for spinning out these things and going and doing it. I just think the first step needs to be like, what does the market really look like? Are you solving a unique hair-on-fire pain? If the answer is yes, go build it internally.
I also think that for larger companies like Dropbox, like Google, like Meta, I actually think I would tweak the philosophy a little bit. They have more resources to give, so I think they should be investing a little bit more on these internal developer tools and then have those teams spin out. I think that's a very healthy dynamic. It gives internal engineers a reason to stay there, to kind of go and prove out this idea and then go spin out. I think that's like a very healthy way to run the startup ecosystem, honestly.
A question from the audience: one of the advantages of rolling your own is solving fiddly corner problems, and the billing space seems like the leader in fiddly corner problems. As a third party, how do you balance the need for that amount of expressiveness that somebody rolling their own might just do?
Amazing question, and you've clearly spent a lot of time in this space. I would say the answer that we had in the beginning was a little bit more like “well we can't solve every fiddly edge case for you.” Like here's the models that we support and if your business model happens to line up directly with us, amazing, Metronome is a great fit for you. As we've grown and scaled and have more and more, honestly, time and money — what we've started to build and what we are building is a system that can encode all of those fiddly edge pieces in a way that you can have your cake and eat it too.
Now as a three person startup we couldn't do that, because that generalized solution space was too out of reach. It's like we couldn't get to market fast enough. But now that we are in market and are serving large clients, we're actually taking the investment money and momentum and funneling it into a platform that allows you to solve your own edge cases. That's been our approach. I think in general with dev tools that is the tradeoff that you make. It's like, is there a good enough solution or do my fiddly edge cases rise to the level of of needing a unique solution?
And I think the one thing that I would encourage people, when you're thinking about this problem, is to consider the short term, but also consider a bit longer down the line. Because what I saw at Dropbox was we would jump in on a problem early and it wouldn't have that many fiddly edge cases, but it would have enough that we were like “let's go build this.” But over time, as the business grew and complexified, the edge cases spun way out of control. The internally built system just couldn't catch up with the needs of the business. So actually a third-party tool, even though it didn't fit us quite as nicely as a glove, solved more of those edge cases in the two to three-year timescale.
I would just encourage you to ask “do the edge cases eventually complexify way beyond your control?” And if they do, I would consider how am I going to handle that fact in two to three years when I have this internal system that my entire company is dependent on. Does that mean we're going to have to staff a five person team to do this, and is the business going to want to do that or not? I think that's the kind of challenge that fundamentally hits a lot of these internally built developer tools.
The most beautiful developer tools are the ones where the problem doesn't complexify in that way. I think if you can identify those, those are the ones where it’s really worth investing all your time, effort and energy. One rule of thumb I like to use is to imagine that, if my company grew to be 10X as big, who's the persona using this product? If it's the same, if it's an engineer, then maybe I can solve it today. If upon success the persona is going to grow and change…
Applicant tracking is a good example. In the early days of Dropbox, the core consumer was like three hiring managers, but over time, if we were going to be successful and grow, it was going to be interviewers, hiring managers, debrief moderators, probably executives and all this stuff.And so I think that problem is fundamentally unsolvable internally because the persona skew, that persona differentiation as the company scaled, was making an untenable long-term solution.
That’s one of the things that we saw when we looked at Dropbox and other super large companies like Twilio. The problem started out as just an engineering problem.Then it quickly became just engineering and sales ops. Then just engineering, sales ops, customer support, finance, accounting- and it grows in this really fast way that means internal systems sometimes can't keep up in the way that you may have hoped or wished.
It’s part of the reason why we wanted to start up a billing company in the first place. We knew it complexified really fast, so that meant we could account for that. It made our jobs really hard, like right now we are like scrambling, scrambling, scrambling to make sure that all of these customers are super happy. But it meant that we felt a proper, externally built product could kind of catch all those edge cases. It doesn't make our job easy, but it does make our job fun because we get to like work with a lot of different stakeholders.
What would you say, more on the engineering side, is a good example of a tool that you love that does that for you guys? Where you're buying into it now because you know, in the future, there's going to be so many stakeholders that are involved with it?
I think one tool that we've been using that I really loved, and I can confess I'm an investor but hopefully not a conflicted investor, is Airplane. Essentially the way I think about it — and probably they market it differently — is as an interface and control plane on top of the scripts that you use to run your business. It gives you auditing control and permissions control over things that otherwise would be run in a shell by an engineer.
What we've found internally as we've used the tool is that you find tons of different use cases for this. Whether it's customer support, whether it's sales, whether it's a one-time data export. It's got this nice, initial, very simple kind of presentation layer, but you can see over time that it's going to grow and become a core component for finance and for sales and for customer support. And you’re like “wow, okay, cool.” It would’ve been really easy, when we just had one script that we were running occasionally, to have engineering be the toil horse on this and run the script whenever customer support needs it.
But having this tool in place lets the engineer write the script once and then update it over time, but then gives customer support the ability to actually invoke and control running of it. It’s like there are multiple users of the thing — there's the engineer who writes the initial script, but then there is an array of people who are kind of consumers of it internally. I felt like that's a pretty elegant solution to this problem.
A question from the Seaplane Discord: how did you communicate internally to direct towards the system’s core value? How were the teams able to understand the core value given the complexity of the system?
I assume this is for Metronome in particular — we kept it pretty focused. Our initial clients were all of a similar size where, truthfully, we were talking to one or two people. So we were talking to an engineer and a product manager so the value prop was very clear. It was like, “don't build the system, we will help grow with you.” As those clients scaled and as we moved up market, I think that core stays the same. But then the trick is we need to be able to explain the vision of how finance’s problems will be taken care of in spite of the fact that you don't actually have a finance person at your company today.
The way we kind of got around that was by saying “here are the people who will eventually be consumers and users of Metronome” and we made sure that we had products and features that were targeted at those consumers. We gave engineering and product the confidence to say “we don't have those problems yet, but I know that we will if we are successful and I know metronome can hold my hand and get us through that.”
I think the secondary thing that we [did was] telling that story and being very upfront about how today the problem is single stakeholder, but in the future it will be multi-stakeholder…Initially we started with smaller companies and then we moved to bigger companies. As we got to those bigger companies, and they had those stakeholders, we had to find the subset of those customers who were excited about the chance to partner with us to really solve their problem.
It’s very collaborative. We go to a finance person and we say, “we don't have the perfect solution for you. In fact, no one does! Look at the market — it’s all terrible for you, but we are here to build with you. Sign up to be a design partner and partner with us and we will do X, Y, Z, and help make sure that this problem is solved for you. We're going to use this as a way of building the value into the product.”
I will confess that was a really hard sale because most people who are in that role are looking for a solution, not for a partnership. We just had to work really hard to find that subset of people who were ready for a partnership and excited about the chance to inflect our roadmap. Now it's a little bit easier because we have these solutions deployed at scale in the wild. For almost any company that comes to us we can point to a 10X larger client who's already using Metronome and we can say, “look, here's how we solve all these problems for you and we can show you the product and features.” But in the beginning it was really about forming deep collaborative partnerships and then spending a lot of time with clients. We call that the “design partner phase” of building up our product and company. It was honestly one of the most fun things. It's the best advice I have for a budding entrepreneur is this design partner concept.
What was the effect for you and the engineering team? If you have a customer that dictates where you're going, and you have your own ideas, what do you build?
It requires a really strong product vision and product muscle. It requires you to say, “okay, I hear what your problems are.” The frame that I would bring to clients was: we're working together. You need to tell me what the most important problems are for you, and my team is going to solve for the market. [We would] explain to them why solving for the market was actually a better solution.
It’s kind of like that prior question about edge cases. A lot of people will come to you and say, “I have this really niche, very specific want of integrating in this exact way and blah, blah, blah, blah, blah.” What we found was if we actually thought about it, and then we just explained to them why this general solution to the problem was actually better for them—
The reason why they believe that, and why it was true, was because their business today is actually fundamentally going to change a lot over the next three to six years. If we have a general solution then the general solution will cover them over time. They won't need to turn off Metronome. A lot of it was just very frank conversations and having a strong product vision that says “look, you want it in a specific way but we're going to do it in this more general way.Here's why that's good for you. Work with us.” Almost always the clients were okay with that.
They wanted these specific things because they had grown up in a specific context and they understood that we understood the broader market and we were driving them toward where the market was going, not where their businesses was at that exact moment.
What do you think is the biggest challenge ahead in cloud engineering?
Great question. I think for cloud engineering…we've kind of said “engineer,” but I think “engineer” is getting to be too general. I think there's going to be a whole new set of engineers. Is a person who spends all day working out of Airtable an engineer? Today, not really. But increasingly that person's going to have, with the amazing amount of tooling out there, the power of a front end or back end engineer. They’re going to have that level of power.
So I think the term of engineer is going to splinter into lots of different profiles. When I think about cloud engineering…I bet you what's going to happen is there's going to be a bunch of niche personas of engineers who are all going to need pretty different things. That’s really exciting from an entrepreneurship perspective because it means that the problem space is just going to grow by orders of magnitude as this concept of “engineer” starts to explode.
I think of like DALL·E 2, [which is] obviously amazing. I think in a world where DALL·E 2 is ubiquitous and cheap…what is a visual designer? How is that going to change? I think the same thing's going to happen in engineering with AI and with data tooling. I think the more tooling we build into the world, the more different types of engineers we get to see, and then that’s kind of recursive. It means that they're going to need more tooling and all that.
To me that's the biggest challenge — that engineering is going to splinter as a concept. I think that's super exciting from a tooling and opportunity perspective.
-
Thank you again to Scott for his time! You can learn more about Metronome on their website, on Twitter, and on LinkedIn.
We host regular events on Twitter (@seaplane_io) so be sure to follow us there and join us for our next live AMA with industry experts like Scott!
