# 349 — April 9, 2021 |
Database Weekly |
HStreamDB: A Real-Time Streaming Database for IoT — A newly open-sourced database designed for working with large scale real-time data streams with standard SQL (plus some stream extensions). There’s a thorough introduction here. It’s built in Haskell. EMQ Technologies Co |
TiDB 5.0: A Distributed HTAP Database — TiDB is a database written in Go and compatible with MySQL’s wire protocol that offers horizontal scalability and aimed at HTAP (Hybrid Transactional and Analytical Processing) workloads. 5.0 adds an experimental list partitioning feature, invisible indexes, and improved ‘desensitization’ of personal data in error messages and logs. PingCAP |
An Operations-Free, Scalable & Flexible Postgres Alternative — Fauna combines the operational integrity and relational modeling of Postgres with an interface and architecture that fits better with modern app development in the cloud. The goodness of Postgres without its operational bottlenecks - Learn more about Fauna. Fauna sponsor |
Database Deep Dives with Andy Pavlo — Andy is an associate professor of ‘Databaseology’ at CMU and certainly knows his stuff! In this interview he explains where he sees database heading in the future, what he’s working on, and self-driving database systems. Josh Mintz (IBM) |
Redis Labs Raises $110 Million in Series G Round — The main company behind Redis is now worth more than $2 billion on paper with this round. They claim to have 31% of the Fortune 500 as customers and deployments continue to go. Their RedisConf event is taking place in two weeks’ time, complete with a hackathon. Datanami |
Quick Bits
|
How GitHub Scaled Its API with a Replicated Redis-Powered Rate Limiter — GitHub’s old Memcached-based rate limiter was simple but running into problems. Here’s how Redis plus some Lua scripting came to the rescue. Robert Mosolgo (GitHub) |
The Three DynamoDB Limits You Need to Know — Those limits are the size of individual documents, the page size limit for Query and Scan operations, and the partition throughput limit. Essential to know when working with DynamoDB at scale. Alex DeBrie |
How to Anonymize Your Data with Amazon S3 Object Lambda — We recently announced the release of S3 Object Lambda, a way to place serverless Lambda functions in the way of S3 object retrievals, and one use case is to anonymize (or at least pseudonymize) data on the fly. Jérôme Van Der Linden |
The Most Important Events to Monitor in Your Postgres Logs — In this eBook, we are looking at the Top 6 Postgres log events for monitoring query performance and preventing downtime. pganlayze sponsor |
Building Real-Time Serverless Data Pipelines for Market Data — This is Google Cloud oriented so technologies on display include Pub/Sub, BigQuery, and Dataflow. Rachel Levy & Bhupinder Sindhwani (Google Cloud Blog) |
Is Amazon RDS for PostgreSQL or Amazon Aurora PostgreSQL a Better Choice for Me? — I think there’s a lot of confusion about when Aurora or straight RDS suit the situation. Here, Amazon attempts to clarify things. Vivek Singh & Sagar Pate (AWS) |
Loading SQL Data Into Pandas Without Running Out of Memory
|
💻 Jobs |
Workday - PostgreSQL Database Engineer / Administrator — We’re looking for an engineer to help us build for the future, revamp our persistence architecture, and ensure our systems are operating flawlessly 24/7. We’re one of the largest SaaS companies globally growing rapidly in Europe. Workday |
Find Data Engineering Jobs with Hired — Take 5 minutes to build your free profile & start getting interviews for your next job. Companies on Hired are actively hiring right now. Hired |
🔧 Code & Tools |
fselect: Find Files with SQL-Like Queries — Do you love SQL so much that you’d like to use its style of code to find files on your system? Your wish has been granted! It goes further than you might think. J H S Petersson |
Google Database Migration Service Now GA — Google Cloud’s Database Migration Service (DMS) is a service for supporting MySQL and Postgres migrations from on-prem to, unsurprisingly, Google’s Cloud SQL service. Shachar Guz (Google Cloud Blog) |
TiKV 5.0: A Distributed Transactional Key-Value Database — Originally created to complement TiDB (featured above) TiKV is inspired by some of Google’s distributed data systems (like Spanner) and not only offers normal key/value storage fare but also transactional APIs with ACID compliance. TiKV Project |
SQLean: All the 'Missing' SQLite Functions — It’s bold to suggest SQLite is ‘missing’ things, but this interesting project adds a variety of new bits and pieces including string, math, and statistical functions. Anton Zhiyanov |
Dflat: Structured Data Store for Mobile — SQLite + FlatBuffers = happiness? Liu Liu |