Learnings from Ente's FOSS and fully private ML

AI models are still growing in capability, size and use every single day. Looking to the future, it's hard to imagine a world where all the benefits of AI don't come at the cost of user privacy. Open-source AI might be thriving, but when it comes to private AI, things look pretty bleak.

At Ente Photos however, privacy is at the core of everything we do. We strive to be a safe haven for people's precious photos, while also making it as easy as possible for them to relive those memories. But to facilitate rediscovery of photos, you need to use AI. So we decided to create a fully private and distributed AI system!

Ente being built on end-to-end encryption (E2EE) from the ground up, our AI (or ML, as we prefer to say) has to run fully local on the user's device. Guaranteeing that no sensitive user data leaves the device unencrypted. Being on all platforms, this mean that our system has to run on every user device. And wanting to provide a seamless experience to the user, everything runs automatically in the background and is then synced (E2EE). None of which is typical, let alone easy, in today's cloud-driven AI world.

In this session, I will walk through Ente's long (for AI standards) journey towards our FOSS and private ML systems. Besides talking about the importance of private AI/ML, I will discuss the possible approaches, what worked for us and what didn't, what technology we're using, and what to consider when building something similar.

For more information on the technical content of this talk, please take a look at our whitepaper on our ML approach. The idea is to give a longer, more technical version of the talk we gave at Mozilla Builders earlier this year.

- Why private and on-device ML is worth consdering

- Ente's journey towards private ML

- Considerations, technology and other practical knowledge for building private ML

- Challenges, solved and unsolved, to creating private ML