[Masterclass] The Power of Artificial Intelligence by Co-founder Ariel Choukroun

Written by Fittingbox | Oct 18, 2024 12:04:57 PM

As part of our "Expert Talks" series, here is a special episode on the Power of Artificial Intelligence provided by Fittingbox's Cofounder and Chief Technology Officer, Ariel Choukroun.

Artificial intelligence is increasingly becoming a hot topic, so we wanted to shed some light on the concept and give a brief overview of the scope of AI today.

About Ariel Choukroun, CTO and Cofounder of Fittingbox

Passionate about the technology behind eyewear virtual try-on, Ariel made it his doctoral subject in 2007 while co-founding Fittingbox with Benjamin Hakoun. He holds a PHD in Computer Vision from the Institut National Polytechnique of Toulouse, after graduating from INSA in Computer Science.

What can we do with Artificial Intelligence?

I'll go straight to the point: you already know there are different types of AI. On the right of the screen, you can see a lot of AI companies coming out these days. And as you know, AI can deal with images, texts, videos, voice recognition and detection.

You all know Google Translate when you go abroad, Google Lens or Amazon Rekognition. And lately, there even has been the topic of generative AI: you all know about ChatGPT for text or MidJourney to create images, or even Sora to create videos. These are some types of AI. The interesting question is: what are the purposes of AI? There are 3 purposes of AI that are interesting and important. There is the analysis and the synthesis capabilities of AI which lead to some deduction. Some people talk about reasoning, but it's not quite there yet.

I'll show you a first example of AI: the Qwen2-VL engine. The goal of this model is to be able to describe a video: you give a video as an input, and it creates automatically a summary of the video. So here, the model understands that it's a video about the ISS. There are astronauts in the video and they are talking with Earth and connecting together. So it's quite an amazing result to be able to analyze and synthesize this huge content.

Another interesting example about huge content is the capability to have a very large input. Here, they talk about tokens: it could be words. The goal of the Claude 2 engine is to be able to summarise articles, papers and books. Here, we give it two articles and ask to give the best highlights of the articles. As you can see, you get all the summary of the articles and the main interesting ideas in seconds. I would say that today you have so much information available to you, that you need this kind of tool and assistance to get a broad vision of something before diving in.

There is also another purpose of AI which is generation. Here is a MidJourney example that creates an image with a prompt. The prompt here is about a an impressionist styled-painting of a Chinese lady wearing a clothing: and you get this kind of very nice and very coherent image. That's for creative purposes, but it helps you to save time for your work usually.

How is Artificial Intelligence used at Fittingbox?

Now, I will talk about how Fittingbox and AI go together.

It's a long story because our first patent on Machine Learning system was back in 2010, and was about detecting face points and eyes points very accurately. Then, we created a strong team dedicated to AI at Fittingbox. Since then, we've never stopped filing patents. For example, we've launched the Frame Removal, which is a nice mix of AI algorithms. The algorithms remove the frame that you are wearing from your face: it erases the frame from the face. To do so, we detect the frame you're wearing and then we can apply the removal, and we do that in live.

If I take how it's working with the input on the left, you have an image and then the task of the AI is to remove the frame. So Fittingbox analyses in 3D what's going on, and as you can see here, you don't have any clue about the color of the temples because you don't see it on the image. With our AI algorithm, we can remove the glasses, but also give the right colour to some elements we didn't see, then we can get the final image. This is only a little piece of the removal technology that I'm showing here, but it's to portray how AI generation is used in a live context.

Now, let me explain how we're going to use AI in the future products of Fittingbox. Nowadays, Fittingbox's R&D works on generated AI to digitize 3D glasses. We also work on improving the analysis of the face and the glasses, and on a very high visual quality and the new rendering engine to be able to get the best results in terms of try-on.

How are solutions based on AI built?

So the best way for us at Fittingbox to make AI and technology work together is the following way: first, we have proprietary algorithms dedicated to eyewear and patented solutions. We've added a specialized Artificial Intelligence with high accuracy and a specialized purpose. We've crafted our data to be the best for getting the best results on a specific task. Now, we can use foundation models to give a general overview and some of the basis for our algorithms.

So, here is a visual foundation model from Meta that gives you some analysis on an image of a user. And at Fittingbox, we aim to analyze a lot of elements very well: the faces, the glasses and so on. That's the kind of results we are expecting and getting right now on our end.

Watch other episodes of our Expert Talks series

View full post