How YouPorn Uses Redis: SFW Edition

简介: I interviewed Eric Pickup, IT Lead at the Manwin group (the company behind sites like YouPor...

I interviewed Eric Pickup, IT Lead at the Manwin group (the company behind sites like YouPorn and Pornhub), to tell us about their transition to using Redis, why they made the switch, and how well it’s worked. Check out Eric’s presentation on Building a Website to Scale, or get started with your own free Redis instance.  

Justin:   Can you talk about when and why you guys made the transition to Redis?

Eric:   Basically, about two years ago we acquired the site [YouPorn]. It was written in Perl at the time, which was one of the reasons I was brought on board. Although I had a history of working with Perl, we quickly decided it was just not feasible to maintain. There just aren’t enough developers around, especially strong senior developers. So, if we were to keep it in Perl it was going to be a pretty stagnant site, which is something we obviously didn’t want to do.

Right away, the decision was made to rewrite it and we started looking at different technologies. Our first instinct was actually PHP, but we didn’t want to limit ourselves so we also looked at Java based solutions. After a bunch of research and looking at what technologies we’ve been experimenting with internally, we decided to stick with PHP.

Previously we’d been experimenting with Redis, Varnish and a few other technologies. Some sites internally had already started to use Redis, mostly as a caching solution, but we wanted to see if we could use it as a real data store.

cat-on-computer

We did some early tests and based our decision mainly on performance, since that was (and is) a huge issue for us. We were very, very impressed with Redis’ general performance, and after some discussion we decided we were going to use Redis as the primary database for the website.

Previously the site had been written in a traditional LAMP stack. It had Linux, Perl, MySQL and Memcached. There were obviously some concerns about the transition. One of the tradeoffs, which I’m actually really glad we did in hindsight, was we kept MySQL in the picture. We don’t read from MySQL on the website, but we are able to use it to do things like populate new lists or hashes, as well as things we couldn’t anticipate ahead of time. We have MySQL more for ad hoc queries, and use Redis for the website.

Soon after we started developing with it, we pretty quickly we felt we’d made the right decision. For the first month or so, we were prepared to reexamine our decision but became comfortable pretty quickly. It was really a good fit for our use case.

Justin:   Why is that, and what were you looking at in terms of evaluating whether or not it was a good decision?

Eric:   Obviously ease of development is a huge one, especially when you are rewriting an entire project like this. Luckily, Redis’ data structures mapped well to what we were doing.

YouPorn at the end of the day is mostly about lists of videos and lists of objects, whether it be comments, favorites, etc. the top rated videos, or the most viewed videos. It’s all lists and then objects, which obviously map well to hashes. We do use some of the other data types but I’d have to say that about 90% of our usage falls into the case of either sorted sets or hashes.

Justin:   After deciding to use Redis, how long did it take to actually implement and have it working?

Eric:   Honestly, back at that point we were still ramping up the team. Like I said, it was a brand new project so it was mostly me and one other person when we did the initial staging.

I’d say within four weeks or so we had a good part of the site prototyped. We had the front page, all the main pages, and most of the video pages done. You could view comments – although at that point you couldn’t add comments – but a lot was done in just four weeks with just two people. This timeframe included learning a new framework (Symphony at the time) so we got up and running pretty quickly.

Justin:   How many instances are you using?

Eric:   I can’t get into specific numbers, but it’s fewer than 10.

Justin:   That’s really impressive. How did you guys manage to have so few?

Eric:   It’s grown over time as we have added functionality, but generally speaking we do a lot of caching with Redis. When we first launched the site, we did no caching. We just relied on Redis.

Over time we found the servers are running a little too hot for our tastes, so we started adding certain levels of caching. We’d have a second Redis node running on the website itself with very short cache times just to handle very popular page views.

You also have to understand that we use Varnish, which sits in front of the web servers so the pages themselves are cached quite a bit so we’re not serving every page via Redis.

Justin:   When you went about making architecture decisions, can you talk about how you decided where to use Redis, and if you made any changes along the way?

Eric:   I’d say Redis was one of the first technologies that we knew we were going to use. That and Varnish, they were early decisions. Our tests on them were pretty good and, like I said, they have already been used by the company before so they weren’t unknown to us.

In terms of what we changed, the biggest change was adding a secondary Redis caching layer. It’s really lowered the queries per second on the servers and allowed us to have more of a safety net there.

Justin:   What would you say the biggest benefit has been after implementing this?

Eric:   For one, I would say the ability to rapidly create new features has been quite powerful with Redis. I mean it’s not just Redis, it’s the full software stack, but we’ve written a nice library that sits on top of the basic Redis libraries which allows us to quickly put together new features. That’s definitely been the biggest benefit we’ve seen.

Justin:   What were some roadblocks or difficulties in making this transition? Was there any custom stuff that you had to figure out and do on your own?

Eric:   Let me think here. Implementing the caching layer took some time. Like I said, the servers were running very hot and we didn’t really want to start throwing more and more servers at the problem, so building a solution took some time.

The other thing that took some time was figuring things out. These days, most websites built using Linux systems are using MySQL as the data store. MySQL does have a huge advantage in that there is lots of documentation. If you run into a problem, chances are somebody has already dealt with it before and you’ll find dozens of sites with information and advice. Redis just doesn’t have that type of community yet. If you want to read testimonials by other people that have set it up and what they’ve learned, what settings they’ve used, what their experiences are, there is a lot less information out there. There are a lot less tips and tricks so there’s more of a learning curve.

There’s just not as much documentation out there compared to MySQL, so finding solutions to issues or simpler things, like setting up replications to disk, took a bit more time. However, as Redis is becoming more popular, the documentation and community is starting to form.

Justin:   Do you have any tips or tricks that you’d like to share with our audience?

Eric:   I’d say most of the most valuable ones, I just don’t know enough about. I’m not a systems man and a lot of it was basically system type stuff. I’d say one trick that’s easy to miss is when you’re setting up replication to disk and you have a cluster of master and slaves, make sure that there is enough time between each one so you don’t end up in a situation where they all decide to write to disk at the same time.

It’s very easy to overlook. Our initials servers were all good but later on when we added more servers occasionally we kept the default settings, which was something we had to fix. It’s one of those things that people could benefit a lot from. I’m a software developer, and I’d say most of the real lessons learned were more at the systems level. I don’t have enough information to really go into those.

Justin:   Great. Thanks for an awesome interview, and hope things continue to go well at Manwin!

Eric:   Thanks for having me.

目录
相关文章
|
7月前
|
缓存 JSON 应用服务中间件
【HTTP】HTTP状态码全分类表(1xx/2xx/3xx/4xx/5xx)(附:《思维导图》)
HTTP状态码是服务器对客户端请求的3位数字响应,依据RFC 7231分为5类:1xx(信息)、2xx(成功)、3xx(重定向)、4xx(客户端错误)、5xx(服务器错误)。本文系统梳理各分类核心码(如200、404、500等),明确语义、场景、特性及常见误区,兼顾规范性与工程实践。
6836 2
|
SQL 数据可视化 数据挖掘
本地数据调用析言的解决方案
本文介绍了通过API创建虚拟数据库,利用阿里云百炼/析言GBI平台实现数据查询、分析及可视化的方法。方案结合本地与云端资源,确保数据安全,同时提供灵活的API调用方式,支持按需调用析言的各项功能,有效降低已有本地数据库系统的迁移成本,提升数据分析效率。
1789 11
「Code Lab科技创新营」 浙江理工大学
蚂蚁金服金融科技携手浙江省内优秀高校与合作伙伴,建立长期的科技育人计划并推出「Code Lab科技创新营」 「Code Lab科技创新营」项目主要开放给具备一定信息技术基础的本科院校三、四年级或在校研究生同学, 邀请到数位蚂蚁金服高级技术工程师及高校博士导师以3天集中课程的形式向学生输出蚂蚁技术与能力,与同学们面对面深入探讨,共同实践。 本次活动以”SOFABoot微服务框架项目开发“为课程主题,由专业老师辅导,学生上手实操开发。真正做到技术不再是理论,上手不再是梦想。
「Code Lab科技创新营」 浙江理工大学
|
3天前
|
弹性计算 人工智能 API
Codex超详细入门指南(下载·安装·登录·上手,全版本覆盖,2026最新版)
建议收藏本教程,2026 全年各版本更新随时回来查阅。如果你在搭建过程中遇到任何问题,欢迎结合 阿里云工单系统 提交技术支持请求,或在 阿里云开发者社区 搜索 Codex 相关话题交流。 祝你编码愉快,效率翻倍!
1670 2
|
6天前
|
JSON 前端开发 API
阿里云最新Qwen4-Max大模型介绍:模型定位、特征、能力、适用场景解析
2026 年 11 月 11 日,阿里发布第四代旗舰模型Qwen4-Max预览版,首发千问 AI 平台 Token Plan,并同步接入千问办公、Qoder 等入口。该版本聚焦知识理解、世界感知、编程与 Agentic 四项升级,主打复杂编程、专业办公、前端设计与专业软件操作。预览期至 11 月 25 日 24:00,到期后模型标识下线;限时 2 折、新用户再享 5 折,最低 1 折可体验。
|
6天前
|
人工智能 前端开发 API
阿里云Qwen4-Max 大模型详解:定位、特征、能力、适用场景与相对Qwen3-Max差异解析
2026年11月11日,阿里发布第四代旗舰模型Qwen4-Max预览版,首发千问AI平台Token Plan并接入千问办公、Qoder等入口。该版本聚焦知识理解、世界感知、编程与Agentic四项升级,主打复杂编程、专业办公、前端设计与专业软件操作等真实任务。预览期为11月11日10:00至11月25日24:00,仅支持Token Plan调用,到期后模型标识下线;限时2折、新用户再享5折,最低1折可体验。
|
3天前
|
Linux Go 开发工具
手机上跑 DeepSeek 智能体:开源安卓客户端 DSH Go 安装与使用指南
DSH Go 是一款开源安卓客户端,将 DeepSeek Harness 智能体引擎装进手机:免 Root、免 Termux,一键安装即用;支持本机运行(内置 Ubuntu+Node.js)或远程接管电脑会话;具备插话打断、过程折叠、屏幕操控、端到端加密分享等特色功能,真正实现智能体“口袋化”。
3314 0
|
19天前
|
Linux 开发工具 iOS开发
Codex保姆级教程(下载+安装+登录+使用,含所有版本,2026最新)
Codex是OpenAI推出的开源AI编程智能体,本地运行,可理解整个代码库,支持自然语言指令改代码、执行命令、跑测试、修Bug,兼容Win/macOS/Linux。(239字)
8589 58
|
13天前
|
Linux 开发工具 iOS开发
2026最新codex下载安装保姆级使用教程(下载+安装+登录+使用,含所有历史版本)
本文详解OpenAI Codex国内快速上手指南:提供Win/macOS/Linux全平台免安装包、npm/Homebrew等多方式安装、账号/API Key双登录、简体中文切换,覆盖下载、安装、汉化、使用及高频避坑(如微软商店失败、npm超时等),10分钟即可跑通。
3042 9