36. Challenge Accepted. with Thomas Payet

Piotr Karwatka: [00:00:00] In today's episode, my guest is Thomas Payet, co-founder of MeiliSearch. MeiliSearch is a powerful, fast open-source search engine. The search engine market is tough, having its own category, leaders like ElasticSearch and others. So my question is, what makes MeiliSearch unique? How are they going to compete with Elastic? And what's the business model? I can't wait to ask all those questions. 

Hi, Thomas. Thank you for accepting my invitation.

Thomas Payet: [00:00:26] Thank you. I'm very happy to be here.

Piotr Karwatka:: [00:00:29] Awesome. Thomas, first, please tell us a little bit about yourself.

Thomas Payet: [00:00:34] I'm currently based in Paris and the co-founder of MeiliSearch. I'm a developer since 2014 and I learned when I was 22 at Ecole 42 (name of school), and what I immediately loved in computer science is the idea I read in SICP that we programers are wizards since we manipulate abstract things called data using esoteric programming languages, the spirits of the computer that will perform intellectual work and subquestion affect our world.

Piotr Karwatka:: [00:01:14] Sounds really awesome. Never thought about what I was actually doing this way, but pretty inspiring. What's MeiliSearch? What is the company about?

Thomas Payet: [00:01:30] MeiliSearch is as Open Source search engine, so we are trying to build a tool that will allow any developers to build such books very quickly and very efficiently.

Piotr Karwatka:: [00:01:43] You found it in 2018, right?

Thomas Payet: [00:01:47] Yeah. So as part of my programming studies, I was working at vente-privee, which is a French e-commerce company, which is well known in Europe for doing flash sales. And I was working in a team called The Lab and we were working on a different innovative project trying to solve problems. The company may face. We were like a little agency within the VP and every few months we were working on different subjects.

Piotr Karwatka: [00:02:13] So at some kind of POC solving some different problems.

Thomas Payet: [00:02:18] Exactly. So at some point we got to work on the search engine because the search bar, the search box is very important to e-commerce and you can very easily grow the conversion rates by 20 or 40 percent if you have a search that is more relevant. We begin to work on this project because at this time when a customer would search for robe rouge, which is red dress in French, he or she would get results around wine, which is not completely false, but very irrelevant since we were actually selling red dresses. So, yeah, we began to benchmark different situation in order to to improve what what was currently existing at this time. We were using ElasticSearch and so we decided to create our own search engine from scratch for four different reasons and then come back to that later. 

Piotr Karwatka:: [00:03:18] How about your co-founders? I guess you you didn’t start your own company solely on yourself. So you've looked at this more on who started it together with you, that would be awesome.‍

Thomas Payet: [00:03:31] So the four of us, we were working in the same time I did. So as I said, I started there. At the end of my studies and I made contact with Quentin de Quelen, who is the CEO today and before he already created a company called Crisp.chat, which is like a lightweight, lightweight alternative to Entercom, which is still performing well. And Clément, who is the CTO, he's a contributor to us so that the language that we are using and has been learning continuously, he is so is profoundly in love with this language. He started to build his first search engine as a side project while we were working together at VP. And yeah, it has been two years and a half that he is constantly looking at how to make it more relevant and faster. So he’s very passionate about this.

Piotr Karwatka:: [00:04:30] Sounds like a dream team. Search engines actually doesn't sound like super easy market. I mean, the challenge itself: improving the strategy is probably pretty complex algorithmically, but also the market is pretty well established with the leaders like Elastic, Algolia, Constructor. Why was the next search engine really needed?

Thomas Payet: [00:05:00] It might seem not very smart of us to enter this market from an outside point of view because. Yeah, I mean, Elastic is one of the biggest open source companies today and ElasticSearch is just shows when it comes to any such related tools to any developers. And yeah Algolia on the other hand is one of the most beautiful French startups and this seems to have a very strong culture. So I mean, it's a very, very nice company too. And they are doing an awesome product. But at some point we found out that, yeah, Elastic is the default search engine to anyone who wants to be the search engine, a search box, and you have to learn how does Lucine, which is the algorithm behind Elastic, how it works. If you want to improve it. And it can be very time consuming, very painful. And we found out that, yeah, Elastic is great, but it's like it has been around for the last ten years and Lucine has been around for the last six years. And we found out that with the new technology such as REST, we would be able to have a more easy to use and simpler experience for developers to build any such works, so that's really what they're trying to build is like something that is as relevant and as fast as Algolia. Yeah, but now with the accessibility, I would say, of open source, meaning that you can trade at any time, you can run it on your own servers that. Yeah, open source is very easy for this.

Piotr Karwatka:: [00:06:51] Gotcha. So let me just paraphrase what you said. So your your aim is at making the search easier, also easier to customize for some cases, make it as far as enterprise grade solution, but still open source. All right, what can be done better in this field? 

Thomas Payet: [00:07:20] I would say, yes, bigger companies than most of us like Spotify, Slack, they are using either Elastic or others, and they are tuning it a lot in order for them to have what we call contextual search sort of such as personalized based on what you search before and how you got.

Piotr Karwatka:: [00:07:43] And this is in the context of the site and the user, exactly what you've done before and maybe what kind of content you have on the website. So this example of searching the dress and getting the wineskin result is a good example.‍

Thomas Payet: [00:08:02] Right, exactly. At some point these are tools today that are only available to you if have the means, if you want to work on these parameters. And it is very stressful for the users. But yeah, your users might not understand why the search on Google or Amazon is working very, very well and why on your website, it might seem a little broken. So, we think that because we are working on it and because we are making it open source, we can really fix all these different sets of practices that could be way better. And yeah, it is just the use of any application and the computer stuff that we are doing every day.

Piotr Karwatka:: [00:08:49] Gotcha, sounds like really strong and good motivation to actually start working on this project. That makes perfect sense. Does it also mean that you built it totally from scratch and it works a different way than all those other things we discussed or do also based on the same algorithms or like? I think that our listeners, including me, are not 100 percent sure and aware. How does the search engine actually work? So if you can share some insights on this. That would be awesome.

Thomas Payet: [00:09:33] Yeah, sure. So how does our search engine work? And it's very basic at the core, is that we have an inverted index containing all the words that link every word there is in your database to every document containing this word. So with this in that index, you can get all documents containing the query you made and from there we are doing a bucket sort. So to sort the documents for the relevancy. So a bucket sort is a sequence of multiple criteria. And between each criterion, we only keep the documents that can be distinguished. So we are not sorting all documents for every criterion. So the search is very fast. So this is basically how the search engine works. So of course, since this, we have a lot of different little optimizations to make it faster and more relevant. But this is basically how it works and it is completely different from what you can read on Lucine, for example, which is, as I said, the search library behind Elastic, which is using what they call the chief IDF term frequency, invest the frequency. So they are basically counting the number of occurrences of your query words into each document, and this is how they get the most relevant documents.

The difference being that when you're using it as if I'm searching for cats and your document contains 10000 times what cats, it will be the first one. And when we are using MailiSearch, we will be looking first in the title and then maybe in the description and maybe the document that would contain the word cats in the title will be more relevant to you than the document containing cats in the description. We did not invent anything, we are trying to get inspiration from the different breakfast and scientific papers that we can find around. Yeah, anything related to search. So basically we learned a lot on the Algolia blog, because they are explaining how the search works. And this is where we started our work and from the area where we are using different research papers. And because we are also using REST, there are a lot of really new and I would say specific implementations that will help us to have some very specific optimization to have a better searcy using the different structures we can use with REST.

Piotr Karwatka:: [00:12:39] Gotcha. That does it also mean that you somehow check what the user actually clicks using some specific phrases like which, you know, to check which results are most relevant for me, like I'm I'm putting this red dress in my search box and the third e results with the best one, does it mean that the next user putting a red dress would get a higher priority on this result? Or there is no such kind of feedback.‍

Thomas Payet: [00:13:12] And that yet actually this is something that, yeah, some people have been asking for and we are considering adding it. We just add some issues regarding the indexation today, because that would mean that we are writing new information to the database for each search, which can be quite heavy. But yeah, this is definitely something we have in mind, because as I said before, this is a kind of contextual search also, because when you're clicking, you can know your ideas at this time. So we can do what you did before. And this can really help improve the relevancy.‍

Piotr Karwatka: [00:13:50] Absolutely. That's a pretty exciting feature, to be waiting for is exciting. 

OK, I haven't seen any notion on that kind of business model pricing on your website or any other materials. When do you plan to monetize the product?

‍

Thomas Payet: [00:14:11] We plan to begin to work on the cloud offering from me sometime this year. This is something some of our different users are asking for. For me a principle is that we are trying to make the life of the developer easy but to free them from just sort of deepening it and maintaining their own version of many such stores. 

Yes, the project is very easy to install, easy to configure, but I am sure you don't want to maintain it in production because of what you should just to be as fast as possible all over the world. At some point, you would need to have your job to get it. And this might be some quite difficult work to do and maintain. So we will do it for the developers. And I think this is really a point where we can add value without harming the open source project, actually.‍

Piotr Karwatka: [00:15:10] Yeah, makes perfect sense, but there is another question that comes to my mind. You probably heard about this, you know, a famous Elastic versus Amazon war let's say. Maybe let's put it like this over using the open source licenses by super cloud providers, so they are making business model out of the actually free software open source software, making millions while Elastic is is actually doing nothing on this business line. Aren't you afraid of being treated like this by these super cloud providers?‍

Thomas Payet: [00:15:52] Yes, yes, of course, this is something we have been thinking from the beginning, and I don't have the solution today. This is a subject I spent a lot of time reading about. And today, thanks to the experience of other companies, there seems to be some way to prevent this. And we can learn from it by reading what happens to the Elastic, and different companie I even begin to make a different license model for this. So. We spent some time and I would say that we are warned. But when we will have this kind of problem and things that will mean that we will be big enough to have different customers. And I really believe that we can have a strong base of customers that would always prefer to buy from the and the contributors of the Open Source project than from just open source vendors. And actually I believe that as an Open Source software company, you cannot capture all the value your open source project can create. So it's also the game when you are doing an open source project and you have to find different ways to try to capture the most out of it. But you could not get all the value from the open source project anyways.

Piotr Karwatka: [00:17:28] Absolutely, I really like and that this problem is, let's say, a landmark for success. If you cross it, it means that the project was really successful. That's that's surely true. But how you finance the product development so far. Do you have some VC’s, investors or maybe it was just, you know, founded by yourself, like the founding team. How was it?‍

Thomas Payet: [00:17:58] So. At some point when we left to work on our own projects on search engine, we were looking for some funds to finance the project because we didn't have any money at this time. And luckily we found and we met Luis Vuitton, at Accattone in our school in France. And they were having a project around search, of course. And because we were working on it for some months, we were present and and with a lot of luck, we wanted to get on. So they became kind of our first customer. So we developed a lot of our search like when we were working for them. And it's maybe paid for that for the first year. And the first two hires that we've got and, yeah one year, one year, I would say, in early in late 2019, we started really to really communicate around the open source project. And and from there, we got some slight tractionon Gihub. And we met some other stars and we convinced them to to help us fund this project. And so so this is where we are today with investors.‍

Piotr Karwatka: [00:19:30] That's awesome. I may be asking one more question in this regards, because I suppose that when you are searching for founding, especially the VC money, there are some priorities. The first and foremost and the most important one is the growth. And I imagine that you can show the eally exponentially growing number of users, 10K stars and growing on Github. And that, by the way, is a really great results. But are they also asking for this revenue money or not yet?

Thomas Payet: [00:20:20] Actually, of course, this is something we are always discussing and we have in mind because in the end, we are a company, so we have to make a difference. But not yet, because the difference in this we are working with and it was not that obvious when we met the first investors in Paris, we had to explain what is Open Source and how does it work. But I think that investors are really getting more and more. And the stable when it comes to open source, so they have to see the point of being like the obvious technology in your own area, so you have to be build the community. You have to have a very strong technology that is just watching. You have yeah. You need to have very strong documentation. You have to get some people using you, even if it's for free. So they have to understand that we cannot be on every side at the same time, on every front, at the same time. So yeah, today the focus is really around making the product better, trying to have the foundation for building a strong community. Because when you're thinking about Elastic, you can see that they have a very, very strong community. Anyone that is talking about it, I think there is always someone coming to you that Elastic is awesome. I think this is really good and this is really something that we would love to have and we begin to have at some point. So we are very happy about it.

Piotr Karwatka: [00:22:05] Gotcha. So till till now, the key focus was the product, the adoption. And as I just mentioned, 10K stars on Github is great success for the project is growing rapidly. What's your recipe for such a success?

Thomas Payet: [00:22:25] We tried different things, and to them, yeah, we are quite happy with the number of stars, of course, and. I think from the beginning, we focused a lot of yeah, we focused a lot on the documentation and the different integration that you can use to do many such. So in the team we have maybe half of the team which is working on the engine and the other half is working on developer accessibility or developer experience from the beginning, even if our documentation is not very dutiful in terms of design, that we really spent a lot of time thinking about how to document and present the story about what is MailiSearch, how to build a narrative about how you can use MailiSearch. And this is something we have very good feedback today about is like a lot of users are saying, yeah, thanks for the documentation. It's very clear and we can use it very quickly. And the second thing is we when it comes to it, I think I agree there are a lot of different integrations in every different languages. And this is something we need to have because, if you as a director, you cannot install and use many such in less than 10 to 15 minutes. That would be a very bad experience for you. So we are trying to have all of these basic languages. So maybe we have 10 different ages today and we are trying to get some more fron-end integration so we can use MailiSearch with React, at some point we will submit to the breaking maybe for WordPress or for Shopify.

‍

(...)

‍