#336 — January 8, 2021

Read on the Web

Database Weekly

Using Checksums to Verify Syncing 100M Database Records — A Shopify engineer thinks through the challenge of rapidly checking that two datastores are in sync using SQL.

Simon Hørup Eskildsen

Presto Poised for a Breakout Year as Data Explosion Continues — Presto is a federated SQL query engine developed at Facebook as a follow-on to Apache Hive and its popularity is growing. Facebook uses it for running interactive queries against several internal data stores, including their 300PB data warehouse.

Alex Woodie

Try DataStax Astra DBaaS | Get Started with 5 GB Free — Start building cloud-native apps fast with Astra, the open-source, multi-cloud stack for modern data apps. With Astra you can build faster with REST, GraphQL, CQL, and JSON/Document APIs. Try it today.

DataStax Astra sponsor

Why CockroachDB and PostgreSQL Are Compatible — Postgres and its wire protocol have continued to increase their status as building blocks of other databases and CockroachDB deliberately implemented Postgres compatibility as a way ‘to be accessible to an ecosystem of drivers, existing code, and developer knowledge’. We’ll see more of this in 2021, I’m sure.

Raphael 'kena' Poss

TimescaleDB 2.0 Now GA: A Petabyte-Scale Relational Database for Time-Series — We announced TimescaleDB 2.0 back in October but it’s now officially GA. What does 2.0 get us? Distributed hypertables, user-defined actions, and a swathe of enterprise features added into the community edition.

Ajay Kulkarni and Mike Freedman (Timescale)

PostgreSQL Is DB-Engines' 'DBMS of the Year 2020'DB-Engines is a popular DBMS knowledge base and each year they have a ‘DBMS of the Year’ based on popularity growth. MySQL had the honor in 2019.

Paul Andlinger and Matthias Gelbmann

Ingesting MongoDB Atlas Data using Amazon EventBridge — EventBridge is an AWS service that provides an event bus for tying together various SaaS apps and other AWS services.

James Beswick (AWS)

Using Amazon DocumentDB's Read Autoscaling — Amazon Document DB (with MongoDB compatibility) is AWS’s MongoDB-(mostly)-compatible document database service and as storage and compute are separated, scaling is really flexible, as demonstrated here.

Randy DeFauw (AWS)

Average Pull Request Merge Time with AskGit SQLAskGit is an open source command line tool for running SQL queries on git repositories.

Patrick DeVivo

New Hard Disk Write Head Analytical Technology To Increase HDD Capacity — Some rather mind boggling research has been taking place that involves imaging the magnetization dynamics of HDD write heads in order to eventually increase hard drive capacity.

Tohoku University

▶  Database Jiu Jitsu: How ScyllaDB Open Sourced a DynamoDB-Compatible API — Corey Quinn (who you might know from Last Week in AWS and his good humored Twitter snark) joined ScyllaDB’s CEO to discuss vendor lock-in, the economics of open source, and what ScyllaDB is up to.


▶  Breakdown Data Silos: Why S3 is the Best Place To Build Your Data Lake — From AWS’s recent AWS Storage Day. It’s basically a 20-minute pitch but ties together a variety of ideas and may be useful if you’re using or considering S3 for anything beyond mere file storage.

Matt Sidley (AWS)

🔨 Code and Tools

Another Redis Desktop Manager 1.4.0 — An open source desktop client for working with Redis that promises it ‘won’t crash’ if dealing with large numbers of keys. Supports TLS, dark mode, and more – seems like the author has put in some serious effort here.


simple-graph: A Simple Graph Database on Top of SQLite — Thanks to its recursive CTE support, you can rig up a graph database on SQLite anyway, but if you’re a Pythonista you might find this abstraction interesting.

Denis Papathanasiou

CYBERTEC PostgreSQL Enterprise Edition: Fully Encrypted & High-Performing — Check out the highly secure PostgreSQL distribution with advanced features like user-friendly monitoring and 24/7 support.

CYBERTEC sponsor

rqlite 5.8: A Distributed Relational Database Built on SQLite — I sometimes wonder if Postgres and SQLite are in a secret fight to have the most number of other databases built on top of them :-) 5.8.0 of this popular distributed database adds support for TLS connections.


Database Lab Engine 2.1: Instant Clones of Large Postgres Databases for Dev Environments — Provision independent non-production environments with multi-terabyte Postgres databases in a few seconds without extra costs.


💼 Job

DevOps Engineer at X-Team (Remote) — Join the most energizing community for developers and work on projects for Riot Games, FOX, Sony, Coinbase, and more.