How Swiggy Built Distance Service - Biweekly Engineering - Episode 6

Welcome back people! Hope things have been going great on your side!

We are back with the 6th issue of the Biweekly Engineering newsletter. In this issue, we have shared engineering blog posts from Swiggy and Pinterest. We also have a Github repository that contains summary of the famous book - Clean Code by Robert C Martin.

Let's go!

Distance Service at Swiggy

#swiggy #microservice #api #caching

Swiggy is a popular online food delivery platform based in India. The company has a huge customer base which is why the systems need to handle large scale of traffic.

In this article, Swiggy shares the design for its Distance Service - a service built on top of Google Distance Matrix API (GDMA) that works as a unified API layer for providing distance and time-to-travel between two points. GDMA is Google service which is capable of providing distance and travel time between an origin and a destination.

Initially, Swiggy started with using GDMA directly. Every system needing distance and time-to-travel data would have their own code calling GDMA. And a shared API key was used. The API had a usage limit - distance and travel time for maximum 1000 origin and destination pairs can be fetched in one second. If this limit was reached, Swiggy would fallback to Open Street Maps (OSM) to fetch the distance information. But OSM's accuracy was not good enough.

A reasonable strategy here is to cache the data received from GDMA on Swiggy's side. This is why the Distance Service was built. Instead of every system going to GDMA for the data, it would ask this new service. The service is responsible for fetching distance and travel time from GDMA as well as caching the response in an in-memory database called Aerospike.

The article has interesting details, like how granular origin and destination points were required (thanks to Geohash), and how they throttled the requests to GDMA.

JSON logging system at Pinterest

#pinterest #logging #monitoring

Logging is an integral part of software development. It gives visibility to the developers on how things are going when the system runs. Logs help debug and find issues as well as create metrics and alerting on top of the them.

Pinterest built an end-to-end JSON logging system to visualize client side app performance in real-time. This is called JSON logging, as the data is schemaless with key-value pairs.

At a high level, JSON logs from Pinterest apps are batch uploaded to the log service. The service relays the messages to Singer - Pinterest's side-car for Kubernetes that manages log uploads to Kafka. These logs are read by Logstash and sent to OpenSearch for visualizing. The system also supports metrics and alerting on top of the logs for better observability.

Logging systems like this make the life of developers easier. Developers at Pinterest adopted the system to visualize network and app errors, performance insights, code execution paths, etc.

A summary of the Clean Code book

#cleancode #books #programming

Clean Code by Robert C Martin is one of the most popular books in the software engineering world, if not the most popular one. The book has a great deal of useful suggestions to write clean code using Java as the language for the examples.

If you have not got the time to read the book, here is an interesting repository on Github. I stumbled upon this on LinkedIn, and found the idea quite useful. The repo summarizes important stuff from each chapter of the book with short examples.

The book itself is highly recommended by literally everyone. This is one of the classical books of software engineering that every engineer should read. If you have not got enough time, at least skim through the notes shared in the repo. The advices very useful.

With that, I am concluding today's issue. It was short and quick. And I hope the content was useful. Don't forget to set aside some time every week to read the stuff I share in the newsletter. I cannot stress enough how beneficial the effort can be.

Take care, and see you soon!