Blueprint of Live Redis Event
Run Effective Meetings with MeetNotes
Free trial for a month
The HashedIn and Redis meetup brings together the cloud computing community to connect, collaborate and learn:
- How to scale-up and boost the performance using Redis
- How to architect internet scaled high availability applications in the cloud using Redis
- Potential to network with other organizations to learn how they are leveraging Redis Labs capabilities
All sessions will be delivered by subject matter experts from Redis Labs and HashedIn, Partners or expert customers can share their real world experiences and lessons learned.
- 09:00 - 09:30: Registration and light snacks
- 09:30 - 09:40: Welcome Note
- 09:40 - 09:50: HashedIn Keynotes
- 09:50 - 10:50: Introduction to Redis Labs
- 10:50 - 11:15: Scale-up and Performance Enhancement using Redis
- 11:15 - 11:30: Break - Tea / Coffee, Networking
- 11:30 - 11:50: MeetNotes: Redis for Real-time Collaboration
- 11:50 - 12:30: Journey of Redis at Myntra
- 12:30 - 12:45 Redis for Cyber Security Enterprises Products
Harshit Singhal, CBO, HashedIn Technologies
A lot of effort went into convincing Redis that India is a prominent market for their product.
Participants must showcase their skills and convince Redis that the market is ripe for their product.
The simile that is used to describe Redis is the swiss army knife
A swiss army knife is a survival tool kit, similarly Redis also has numerous applications.
How one uses Redis is important and it is important to examine whether Redis is used effectively.
Himanshu Varshney, CEO, HashedIn Technologies
- A brief introduction into how HashedIn came into play. The first thing that is written in code is “ # include” and “hash out” is often is used to describe “solving a problem: and that is where the name HashedIn comes from.
- HashedIn is “Wired for Speed” and we translate ideas into products in weeks, not years.
- Founded in 2010 with the mission to help businesses in Product development..
- In 7+ years, we have built, transformed and launched 90+ products for customers in US and APAC.
HashedIn is involved in:
- Digital Transformation
- IoT Enabler
- Artificial Intelligence
- Machine Learning
- Smart Cloud
- Voice Systems
What we do::
- Digital Strategy Consulting
- DevOps Automation
- Smart Platforms
- Customer Experience
- Data Engineering
- IoT Driven Systems
HashedIn and Redis - Connecting the Dots
- 2009 - Sripathi (HashedIn, CTO) started contributing to Redis
- 2013 - HashedIn created Redis Center of Excellence (CoE)
- Redis has been an integral part of all Digital Solutions
- Tech Consulting - Helping businesses in using Redis the right way
- 2017 HashedIn became Redis Labs Partner
- 25+ Experienced Redis Engineers
- Only Emerging Tech Partner for Redis Labs in APAC market
Introduction to Redis Labs
Amir Levy, Director of Sales, Redis Labs
- Redis is Open source. The leader in-memory database platform, supporting any high performance operational, analytics or hybrid use case.
- Redis Labs is the open source home and commercial provider of Redis Enterprise (Redise) technology, platform, products & services.
- Redis tops database popularity rankings
- Redis 4.0 allows you to extend Redis functionality and most commonly used Modules are Neural Redis, RedisML and RediSearch.
- RediSearch is 5X faster than Elasticsearch because it uses Redis in the underlying architecture.
- GST calculations used Redis cache for intensive calculations.
- Redis is state of the art database, opensource and proprietary tech built in multi tenancy, offering effortless scaling.
- ACID compliant, high availability.
Tal Shkolnik, Solution Architect Expert, Redis Labs
Tal Shkolnik, gives a demo of Redis enterprise version.
Scale-up and Performance Enhancement using Redis
Sripathi Krishnan, CTO, HashedIn Technologies
Redis is mostly used as cache. Several use cases exist where features can be built on top of Redis. It can solve scalability and performance issues.
Use Case #1: Real Time Analytics-Dashboard
- Track KPIs in real time
- Cohort analysis
- Leader boards
Use Case #2: Session / API Tokens
- Logout if idle v/s logout after fixed time
- Redis Log out based on inactivity
- Force logout if necessary
- Reduce load on your database
Use Case #3: Task Queues for Background Jobs
- Github uses
- Distribute load to background jobs
Use Case #4: De-duplication Within Time Window
- Duplication occurring within short time window because of retries.
Use Case #5: Rate Limiting
- Redis can track number of requests per minute/per hour, this can be used to prevent overloading resources.
Use Case #6: IP Blacklisting
- If you detect rogue ip addresses, you can blacklist them in Redis. Nginx can then access Redis and ban the IP addresses.
- Achieved by use of Lua.
Use Case #7: Feature Gates
- Store settings on Redis which can be accessed instantaneously.
- This avoids the need to restart server.
Use Case #8: Real-time Notifications
- Redis pub/sub let's you eliminate polling in JS.
- Using libraries like socket.io you can build real-time collaborative apps
Use Case #9: Autocomplete API
- Redis Sorted Set is the perfect tool to build autocomplete APIs.a
Q. ActiveMQ for user notifications vs Redis
MQ based products RabbitMQ and ActiveMQ are more robust but Redis has most of the features of AMQP and can be used for numerous scenarios.
Q. Does expiration work on nested keys?
Nested keys cannot be expired, only top level keys have expiration.
MeetNotes: Redis for Real-time Collaboration
Anshuman Singh, Founder, MeetNotes
Introduction to MeetNotes and how Redis is used in MeetNotes.
Meetings Love to Hate Them
37 billion dollars are wasted on meetings in the US alone and 15% of organization time is wasted in meetings. This is the problem that MeetNotes is trying to solve.
- MeetNotes is a real time collaborative document.
- It parses information as you type or speak.
- It creates action items with due dates, tags and enables relentless follow-ups.
Redis @ MeetNotes.co
Redis is used
- As a cache
- As a Task Queue
- For Real time collaboration
- Operational Transforms are used to provide consistent outcomes and to merge changes
- It Preserves user intent and converge to common state
- Redis Pub-sub helps in inter-user communication
- Relay changes
- Do it reliably & fast
Q. How are concurrency issues handled?
Operational Transform layers take care of that.
Q. How scalable is the infra?
A single machine is capable of handling approximately 100 socket connections but that does not take care of data operations(operational transform).
Q. Long Polling vs Sockets
Sockets help in bidirectional flow. Single socket takes care of both pub and sub. Further sockets deliver results instantaneously.
Journey of Redis at Myntra
Sanjay Yadav, Principal Architect, Myntra
Myntra uses redis for more than 200+ services and there are over 400 instances of redis (Cluster / Sentinel / Twemproxy).
- How it Began in 2011
- PHP Stack
- Checkout experience was shaky.
- Customer location serviceability - Unpredictable
- High latency across multiple pages.
- Customer will not stick if low latency is not guaranteed.
What Was Missing
- A distributed fast access store.
- Stores everything in memory.
- Key value store with expiry.
- Data structure store.
- Popularity knows as "Memcached on steroids".
- Master slave
- Key value store
- JSON Values
- 10,000-20,000 concurrent users.
- Up to 100 k rpm with ~25 ms response time
- In-house client built with auto failover mechanism.
- Switches slave to master when master is detected as down upon 3 successive pings.
- The issues of horizontal scaling were not yet addressed.
- All request were going to a single box.
- Scale is increasing .
- Customer base is increasing.
- In pursuit of growth events are being organized - load increases further.
- Frequent outage on many servers including redis boxes.
- Multiple Nodes
- Sharding logic
- One proxy and 4 master
- Blazing fast.
- Able to support high concurrency - Twempoxy claimed billion request/sec on single instance.
- Horizontally scale redis.
- 1 Million rpm achieved without much efforts.
- It became the single point of failure.
- When throughput increases, bandwidth on twemproxy boxes gets consumed.
- Mget was missing from twemproxy.
- In order to remove SPOF, we created multiple proxies but that leads to inconsistencies.
2015 Redis cluster
- Seamless multi-master setup.
- Seamless fail-over from master to slave on failures.
- We need not worry about shard management. no inconsistencies, intelligent clients.
- Truly horizontal scaling.
- Served 30M rpm at 95th percentile 25ms in a recent event Single instance gave us 50K rpm
Session - Biggest Use Case
- Every request that reaches us has to be authenticated.
- Cumulative of all request we serve.
- Single key, value store.
- 600K concurrent users come to Myntra at event peaks.
Sentinel for Master Slave Setups
- When you don't need too many write servers and need multiple read servers you need horizontal scaling only for reads.
- Cases where management is very important, need all keys, to be available in a single box.
- Setup one master and multiple slaves as per throughput requirements.
- Reads were sent to slaves.
- Myntra social platform - Look good platform
- Users, objects and actions - first class citizens
- Objects can be published by any external apps to the platform
- Users perform actions on other user/objects.
- Feed is created for users based on interest.
- 10 Million Weekly Active Users 500K objects added everyday 100K actions per minute..
Redis Has Always Helped
- Powerful data structures.
- Group multiple keys under one key.
- Store unique sorted values.
- Simple increment/decrement operations.
- Feed generation using Celery - RabbitMQ + Redis.
- Style chat with celebs like Hritik/ Deepika/ Virat Kohli.
- Launch of new brands with brand engaging users
Redis for Cyber Security Enterprise Products
Anindya Sharma, Co-Founder and CTO, Cyware Labs
How Redis helped solve critical issues and improved performance for Cyware and its customers.
- Main caching choice for all layers.
- ElastiCace -> Local Cache -> Global Cache
- Persistence (across deployments and restarts).
- Smart invalidations.
- RPUSHX Update only existing keys.
- Overall RDS cost went down by 30%.
- Reduced dashboard load time.
Cyware Feeds Broadcasting Pipeline.
- Crawler to clients pipeline.
- Cushioning based on component capacity.
- Cy-bank channels.
- Client apps can subscribe to the channels.
- Auto publish based on client’s wish.
- Better capacity handling at clients.
Near Realtime Security News Crawling
- Real time Security
- New Crawling
- Crawling more than 10,000 security news websites and blogs every 30 mins
- Repetitive crawling of news that has been published a few minutes back
- Real time updating