15. Challenge Accepted. with Eli Finkelshteyn

[00:00:42] Piotr Karwatka: In today's episode my guest is Eli Finkelshteyn, founder of Constructor.io. Constructor is an e-commerce search engine used by Sephora and other top retailers for improving their conversion rates. It's pretty unique because it's actually optimized to just sell more. Previously Elie’s career was all about the Big Data at Tumblr and Shutterstock. Hi! Thank you for accepting my invitation.

[00:01:07] Eli Finkelshteyn: My pleasure. Thank you for having me. 

[00:01:09] Piotr Karwatka: Elie, how your career started? It seems that you began with a rather academic path.

[00:01:16] Eli Finkelshteyn: A little bit. I got into computers when we got our first one around middle school and then in college, I got really interested in linguistics and human languages and I've kind of been trying to find ways to work on both of them at once ever since. So at first that was academia. I was doing computational linguistics mostly in university. I did a triple major in math, economics and computer science and then kind of gravitated towards computational linguistics. But then I realized you can have a much bigger effect on the world outside of the classroom. And you probably run into this as well. I mean, our servers power like a few billion requests a month, and that's just not something that you would ever be able to do or find in academia. The satisfaction of having that impact is, for me at least, the key driver. 

[00:02:16] Piotr Karwatka: Gotcha. So you got into big data very early on. Do you remember your first project outside of academia?

[00:02:24] Eli Finkelshteyn: I don't remember my first project outside. Well, it's funny. I can tell you about my last one in academia. And then I'll tell you about... Give me a second to think about the first one outside of academia.

[00:02:43]  Piotr Karwatka: Sure, go on!

[00:02:46] Eli Finkelshteyn: My last one in academia was... I'm telling you just because it's something I haven't been able to work on in a long time. It was something that I really enjoyed. I was doing a speaker sentiment analysis for my master's thesis in Aramaic. So I've had a thing for ancient languages for a while and I figured that was going to be my last chance to ever work on them. Cause you can't really find a job outside of academia doing that. So that was a lot of fun. And it actually reminds me of my first project outside of academia, which similarly enough was also on something like sentiment analysis, but it was in the financial space. So I was working at a company called Fact Set at the time. And one of the important things that we needed to do at Fact Set was we had all of these analyst reports about stocks that were coming in. And the faster that you could realize the gist of what an analyst report was saying, if you could realize that faster than anybody else could, then you could take action on it faster than anybody else could. And so the traditional ways were of course, like a human looking at this and parsing it. But you could actually get to something that was very close to that same level of quality faster, but using machine learning on top of these things. 

[00:04:08] Piotr Karwatka: Wow. Both projects actually sound like pretty difficult challenges because for this Aramaic I guess that you needed to create the whole model of the language itself. For English you can find some libraries, data you can use for modeling the grammar and all the other aspects. For Aramaic I guess not.

[00:04:32] Eli Finkelshteyn: Yeah, I wasn’t naive, as many college students are. It was a lot of fun, the project itself, but it turned out to be much harder than I expected it to be. To your point, like you have to figure out parsing air, maybe doesn't have punctuation or anything like that. It makes the problem more interesting, but yeah, it was fun. 

[00:04:56] Piotr Karwatka: Sounds like something you can really dig into and get all the fundamentals, how they work. So actually, it's a very important lesson, I guess. What else have you learned working on Tumblr, ShutterStock, the Black Plane and your own consulting shop that were your next milestones or what have you learned during this time working for those companies?

[00:05:24] Eli Finkelshteyn: That's a good question. I guess the biggest thing that I learned was just about what makes me happy in a job. 

[00:05:37] Piotr Karwatka:  What is it? 

[00:05:38] Eli Finkelshteyn: So I'll put it this way. It's like the thing that drives me at least is having the greatest impact. And I always knew that I'm just trying to think of the right way of phrasing it. I mean, I think that's right. It's just learning more about yourself through doing these different things, to find out what makes you happy. Cause you spend so much time at work. And if you're not doing something that makes you really, really happy, that makes you excited to wake up in the morning and check what you're going to be working on that day. Then it's just kind of sad. And so that was I guess the biggest thing that I learned.

[00:06:17] Piotr Karwatka: I like this answer.... Because otherwise you actually might not have enough time to start being happy.

[00:06:24] Eli Finkelshteyn: Yeah. If you're only being happy outside of work, that’s hard to find enough time to be happy.

[00:06:34] Piotr Karwatka: Exactly. Okay, perfect. How did the idea of starting a new venture come to you? 

[00:06:41] Eli Finkelshteyn: So, funny enough, I was working on search engines for a while before Constructor.io. So I started working on search engines at a company called Shutterstock. They actually approached me through Tumblr. And the reason I wanted to try working there at all was because I got to work on search. The cool thing about search for somebody who's a little bit of a competition linguistics nerd is it's really got everything. I mean, if you like computer science and you like linguistics, you've got all of the most interesting algorithmic problems that you see in computer science. You've got all the really interesting machine learning problems. If you're doing it across multiple languages and you're doing a whole bunch of NLP, then you've got both linguistics and the computational linguistics problems. And kind of best of all, none of them have been solved. Like you still see Google even right now, publishing papers on new things that they've invented in search. And these were like thousands of the most intelligent people in the world that have been working on this stuff for 20 plus years. And the thing that got me into it was I was working on search at Shutterstock. And then I was doing a little bit of that same stuff consulting. And the thing that you really realized is you start working on search is, first of all, it's an unsolved problem. We already knew that, but. The second one that's really cool is you can have a huge impact. You can literally AB test when you make changes to see how much more of whatever it is the company cares about usually revenue that you're making for them. And knowing that I had that impact, knowing that I could cause that kind of impact, that was incredibly satisfying. But at the same time, this realization hits you. That's like: you're working on this stuff with maybe a team of a few people and in the meantime, Amazon and Google, I mean, those guys have hundreds, thousands of people working on this. And it kind of feels like you're inventing arithmetic when they already invented calculus. Like on the one hand, you're satisfied that you're doing this thing that causes revenue growth. But on the other hand you're really reinventing the wheel. So the idea we wanted to start Constructor.io was, we figured: let's just do it once. We'll do it well, and then nobody ever has to do it again. And we'll let all of these other companies compete with it. 

[00:09:12] Piotr Karwatka: Yeah, that makes perfect sense. So what was the key goal? The key problem you wanted to solve starting the Constructor.io? If not competing, not fighting with Amazon and Google, what was the key goal? By the way, today I was recording another interview with Boris, the founder of Spriker e-commerce platform. And he told me that the biggest lesson he learned along his professional career was: “to wisely choose the fights you fight”. You, you cannot win all the fights. You need to choose wisely... You chose search for e-commerce. So what was the key goal, the key problem you wanted to solve initially at Constructor.io?

[00:10:05] Eli Finkelshteyn: Yeah. And I actually liked that response from Boris quite a bit. So, we knew that the eventual goal was we wanted to let all of these other e-commerce companies be able to compete with the likes of Google and Amazon. But to get there, you really need to build search from scratch. You can't just build it on top of Solr Elasticsearch because then you eventually plateau way too early. I think a lot of people have learned that lesson. I learned that lesson.

[00:10:34] Piotr Karwatka: That's interesting. What do you mean you plateau too early? That you are satisfied with the results and you're not investing any more?

[00:10:41] Eli Finkelshteyn: There's a fundamental algorithm that sits at the bottom of both elastic search and Solr. It actually sits on the Lucene layer. All of that stuff basically assumes that the fundamentally right way of doing search is an algorithm called TF IDF. Which you're probably familiar with, if you've done any kind of machine learning or especially in LP, you're probably familiar with it.

[00:11:06] Piotr Karwatka: I know how it works, but maybe you can explain it a little bit for our listeners. 

[00:11:11] Eli Finkelshteyn: Yeah. So it stands for term frequency, inverse document frequency. It's actually an algorithm that's been around for, if I remember it, longer than computers, it was invented in the 1950s or something like that. And the basic idea of it is to figure out how important a word is or a set of words are to a document. You're going to be looking at how frequently they appear in that document. So if they appear more frequently than they're going to get a higher value. And then you're also going to look at how infrequent they are across a set of all documents. So if something is very infrequent across the set of all documents, but is very frequent in a certain document, then it's going to be considered very important. 

[00:11:58] Piotr Karwatka: And by checking this vocabulary like general vocabulary frequency, you are removing all the punctuation and other stuff, right?

[00:12:06] Eli Finkelshteyn: Exactly. The problem with it is like, it's a very good general purpose algorithm. If you don't have access to, let's say behavioral data, like it gets you okay results, but it's never going to get you great results. And it makes this fundamental assumption that the way that product discovery should work, that search should work, is based of keyword matching. That's just something that we disagree with. Like really, you should let voters vote with their clicks, but if you want to do that, it turns into a machine learning problem. And you can either build that as a layer on top of Solr or Elasticsearch and still wind up kind of being tied to those algorithms that you don't really want any more that TF, IDF, or BM25 base, or you can build it that way from scratch. And if you look at the Googles of the world, that's what they're doing. That was the right thing that we thought to do.

[00:12:56] Piotr Karwatka: So you have built everything from ground up, right? You haven't relied on those libraries you can find, it actually starts with building this from scratch. 

[00:13:06] Eli Finkelshteyn: I put it this way. I mean, it's not that we don't use open source. We use plenty of open source. It's just this one really important piece that everybody else uses open source for the Lucene Solr Elasticsearch part of it. That we built from scratch. 

[00:13:21] Piotr Karwatka: Yeah, it makes sense. Actually, this also led me to the next question. Because everyone says that the product is AI, ML and machine learning base, what does exactly mean for Constructor.io?

[00:13:37] Eli Finkelshteyn: Yeah, that's a good question. We run into the same thing all the time, and I think we ran into this early on, where, when we first started, there were barely any company, not barely any, but there weren't a whole lot of especially product discovery vendors that were talking about the AIML piece. I don't want to name any names, but I can list a bunch of our competitors that weren't really talking about that stuff. And we started to do it and we started to have positive effects in getting some pretty good logos. And then all of a sudden we were looking at our competitors websites and you start to see AI and ML appear everywhere. Somebody who is intelligent in marketing realized that they could just slap a sticker on there that said AI or ML. And of course it's a black box, then, how can you tell the difference? So for us there were really two parts to it. The first one is, just being effective. Like our customers, they could AB test us. And at this point we've had a B test against just about all of our competitors and we consistently win them. And so it's like, you know, traditionally with this sort of thing, you might do a feature sale where you try to talk about all the great features that you have, and that stuff's important, but to us, it's not the be-all end-all, the be-all end-all is can we earn our customers more revenue, more profit, more, whatever it is that they care about than anybody else. And if we can do that, then we feel successful as a company, we feel like we're doing the right thing. And if we can't, then we feel like we've gone astray. The second one, is just looking effective. You turn that black box of AI where somebody else will say like, “oh yeah, you can't see what's happening in here, but I promise it's happening” and it's great. What you do instead of that is you open it up. You show people the actual algorithms, you show them how it works. You let them see it for themselves. You let them see how it's changing. It does two things. The first one shows that we're actually doing this stuff so they don't have to actually just take our word for it. They can see it. The second one is from the merchandising perspective, it lets them understand why we're doing what they're doing so that they can be smarter about what they do in response.

[00:15:48] Piotr Karwatka: Okay. Makes, makes perfect sense. Thank you for this answer. I also saw a demo of a very cool feature. You have guys is it's voice search. Can you say something more about this feature and how it works? I think it's pretty unique. 

[00:16:06] Eli Finkelshteyn: I appreciate that. It was a lot of fun to work on. One of the reasons that we went into it was there was a report from Google, I think in 2016, way back in 2016, that was something like 20% of their mobile search at that point was coming over voice.

[00:16:25] Piotr Karwatka: Wow, it’s a pretty huge number. 

[00:16:26] Eli Finkelshteyn: It's a pretty huge number. They haven't published anything about it since, so we don't really know what the number is now, but it was increasing pretty steadily. And we figured, why is that not the case elsewhere? And it's because, with Google, you can actually trust that the voice search works well. You try this at most of the websites, it's like a gimmick or something like that, it doesn't really work well. And so people don't come back and repeat use it. So what we wanted to do was we wanted to do something that didn't just demo well, but it actually worked well in the way that Google's works well. So that if people used it, it actually created a real customer benefit. So it was a lot of fun to work on. I mean, selfishly it got me back into some of that like linguistic stuff that I was really passionate about. So two birds with one stone, I guess. 

[00:17:15] Piotr Karwatka: Yeah. I think that this is cool that Constructor.io is pretty much domain oriented, working very closely with the database of products you have in your store. So I guess it very positively impacts the accuracy of all those features. Not only in voice search but the search in general, right? Because you know your terms, your database of words, of entities, everything very, very well. It’s a huge benefit, right? We're having this domain specific approach and focusing on e-commerce, so not doing the general thing, like Solr and Elastic. I really like this idea.

[00:18:01] Eli Finkelshteyn: Yeah, we think so. I appreciate that. 

[00:18:04] Piotr Karwatka: That's cool. What are you actually selling at Constructor.io? For me you are selling the CRO conversion rate optimization. Like this is the ultimate value, right? From using Constructor.io as a search. 

[00:18:21] Eli Finkelshteyn: Well, I’d pushed back on that a little, but finish your question first.

[00:18:25] Piotr Karwatka: Okay. So the question was if you have any insights to share how much you can actually improve the CR (conversion rates) over the search experience, you said you are doing a lot of AB testing, so I'm just curious. 

[00:18:42] Eli Finkelshteyn: Yeah. I'll answer it in two is the first one is just like a slight modification, which is we can do CRO, but it's not all that we do for us if you're doing search correctly, then you're treating search as a machine learning problem, mostly supervised machine learning problem. And if you're doing machine learning, then you've got some Y variable, some metric that you're trying to optimize for. And that can be a conversion rate. If that's the most important thing, you can absolutely put that in there, but the beautiful thing about building this way is you don't have to make it CRO. You could make it revenue if you wanted to, you could make it profit if you want it. It's whatever's most important to you as a business. So depending on which customer we're working with, depending on which metrics are the most important to them, you'll see different outcomes. Generally the one that we see most often is some variant of revenue. RPV we see pretty frequently. And the goal for us is always regardless of what we're charging the customer, we want to always make them at least 10 X, what they're paying us. If we can make them 10 X, what they're paying us and they can look at this and they can say “I made a really, really good decision to go with Constructor.io” and like, you want to give them all the great features on top and everything like that. But at the end of the day, when you're evaluating, when the, whoever has given us the trust or evaluating if this is a good idea or not, if they can go to their boss and say: “Hey, I made 10 X ROI on this thing”. No, one's going to get upset about that. 

[00:20:14] Piotr Karwatka: Of course it makes this decision to buy Constructor.io a no-brainer, I really love it. It's a very clear value proposition. 

[00:20:27] Eli Finkelshteyn: One other thing I'll mention on that is that it's less about the value proposition for us, and it's more about just repaying the trust afterwards. That's the main... I mean, you can sell on a whole bunch of different things to get somebody into the door. But like these people, they trusted us. They trusted us with one of their major initiatives for the quarter. They trusted us with really a whole lot of their cash. And we want to make sure that afterwards they can look back on it and say, this is a great experience. 

(...)

‍