# 349 — April 9, 2021

Read on the Web

Database Weekly

HStreamDB: A Real-Time Streaming Database for IoT — A newly open-sourced database designed for working with large scale real-time data streams with standard SQL (plus some stream extensions). There’s a thorough introduction here. It’s built in Haskell.

EMQ Technologies Co

TiDB 5.0: A Distributed HTAP Database — TiDB is a database written in Go and compatible with MySQL’s wire protocol that offers horizontal scalability and aimed at HTAP (Hybrid Transactional and Analytical Processing) workloads. 5.0 adds an experimental list partitioning feature, invisible indexes, and improved ‘desensitization’ of personal data in error messages and logs.


An Operations-Free, Scalable & Flexible Postgres Alternative — Fauna combines the operational integrity and relational modeling of Postgres with an interface and architecture that fits better with modern app development in the cloud. The goodness of Postgres without its operational bottlenecks - Learn more about Fauna.

Fauna sponsor

Database Deep Dives with Andy PavloAndy is an associate professor of ‘Databaseology’ at CMU and certainly knows his stuff! In this interview he explains where he sees database heading in the future, what he’s working on, and self-driving database systems.

Josh Mintz (IBM)

Redis Labs Raises $110 Million in Series G Round — The main company behind Redis is now worth more than $2 billion on paper with this round. They claim to have 31% of the Fortune 500 as customers and deployments continue to go. Their RedisConf event is taking place in two weeks’ time, complete with a hackathon.


Quick Bits

  • Intel has unveiled a new 10nm 'Ice Lake' CPU aimed at the datacenter and AI and cloud workloads in particular. 40 cores and 8 memory channels should be enough to tear through your data.
  • Not really a database story but Google has succeeded over Oracle in the long running Java API case. Anyone mimicking Oracle's database APIs may now also be breathing a sigh of relief.

How GitHub Scaled Its API with a Replicated Redis-Powered Rate Limiter — GitHub’s old Memcached-based rate limiter was simple but running into problems. Here’s how Redis plus some Lua scripting came to the rescue.

Robert Mosolgo (GitHub)

The Three DynamoDB Limits You Need to Know — Those limits are the size of individual documents, the page size limit for Query and Scan operations, and the partition throughput limit. Essential to know when working with DynamoDB at scale.

Alex DeBrie

How to Anonymize Your Data with Amazon S3 Object Lambda — We recently announced the release of S3 Object Lambda, a way to place serverless Lambda functions in the way of S3 object retrievals, and one use case is to anonymize (or at least pseudonymize) data on the fly.

Jérôme Van Der Linden

The Most Important Events to Monitor in Your Postgres Logs — In this eBook, we are looking at the Top 6 Postgres log events for monitoring query performance and preventing downtime.

pganlayze sponsor

Building Real-Time Serverless Data Pipelines for Market Data — This is Google Cloud oriented so technologies on display include Pub/Sub, BigQuery, and Dataflow.

Rachel Levy & Bhupinder Sindhwani (Google Cloud Blog)

Is Amazon RDS for PostgreSQL or Amazon Aurora PostgreSQL a Better Choice for Me? — I think there’s a lot of confusion about when Aurora or straight RDS suit the situation. Here, Amazon attempts to clarify things.

Vivek Singh & Sagar Pate (AWS)

Loading SQL Data Into Pandas Without Running Out of Memory
Itamar Turner-Trauring

💻 Jobs

Workday - PostgreSQL Database Engineer / Administrator — We’re looking for an engineer to help us build for the future, revamp our persistence architecture, and ensure our systems are operating flawlessly 24/7. We’re one of the largest SaaS companies globally growing rapidly in Europe.


Find Data Engineering Jobs with Hired — Take 5 minutes to build your free profile & start getting interviews for your next job. Companies on Hired are actively hiring right now.


🔧 Code & Tools

fselect: Find Files with SQL-Like Queries — Do you love SQL so much that you’d like to use its style of code to find files on your system? Your wish has been granted! It goes further than you might think.

J H S Petersson

Google Database Migration Service Now GA — Google Cloud’s Database Migration Service (DMS) is a service for supporting MySQL and Postgres migrations from on-prem to, unsurprisingly, Google’s Cloud SQL service.

Shachar Guz (Google Cloud Blog)

TiKV 5.0: A Distributed Transactional Key-Value Database — Originally created to complement TiDB (featured above) TiKV is inspired by some of Google’s distributed data systems (like Spanner) and not only offers normal key/value storage fare but also transactional APIs with ACID compliance.

TiKV Project

SQLean: All the 'Missing' SQLite Functions — It’s bold to suggest SQLite is ‘missing’ things, but this interesting project adds a variety of new bits and pieces including string, math, and statistical functions.

Anton Zhiyanov

Dflat: Structured Data Store for Mobile — SQLite + FlatBuffers = happiness?

Liu Liu