#336 — January 8, 2021 |
Database Weekly |
Using Checksums to Verify Syncing 100M Database Records — A Shopify engineer thinks through the challenge of rapidly checking that two datastores are in sync using SQL. Simon Hørup Eskildsen |
Presto Poised for a Breakout Year as Data Explosion Continues — Presto is a federated SQL query engine developed at Facebook as a follow-on to Apache Hive and its popularity is growing. Facebook uses it for running interactive queries against several internal data stores, including their 300PB data warehouse. Alex Woodie |
Try DataStax Astra DBaaS | Get Started with 5 GB Free — Start building cloud-native apps fast with Astra, the open-source, multi-cloud stack for modern data apps. With Astra you can build faster with REST, GraphQL, CQL, and JSON/Document APIs. Try it today. DataStax Astra sponsor |
Why CockroachDB and PostgreSQL Are Compatible — Postgres and its wire protocol have continued to increase their status as building blocks of other databases and CockroachDB deliberately implemented Postgres compatibility as a way ‘to be accessible to an ecosystem of drivers, existing code, and developer knowledge’. We’ll see more of this in 2021, I’m sure. Raphael 'kena' Poss |
TimescaleDB 2.0 Now GA: A Petabyte-Scale Relational Database for Time-Series — We announced TimescaleDB 2.0 back in October but it’s now officially GA. What does 2.0 get us? Distributed hypertables, user-defined actions, and a swathe of enterprise features added into the community edition. Ajay Kulkarni and Mike Freedman (Timescale) |
PostgreSQL Is DB-Engines' 'DBMS of the Year 2020' — DB-Engines is a popular DBMS knowledge base and each year they have a ‘DBMS of the Year’ based on popularity growth. MySQL had the honor in 2019. Paul Andlinger and Matthias Gelbmann |
Ingesting MongoDB Atlas Data using Amazon EventBridge — EventBridge is an AWS service that provides an event bus for tying together various SaaS apps and other AWS services. James Beswick (AWS) |
Using Amazon DocumentDB's Read Autoscaling — Amazon Document DB (with MongoDB compatibility) is AWS’s MongoDB-(mostly)-compatible document database service and as storage and compute are separated, scaling is really flexible, as demonstrated here. Randy DeFauw (AWS) |
Average Pull Request Merge Time with AskGit SQL — AskGit is an open source command line tool for running SQL queries on git repositories. Patrick DeVivo |
New Hard Disk Write Head Analytical Technology To Increase HDD Capacity — Some rather mind boggling research has been taking place that involves imaging the magnetization dynamics of HDD write heads in order to eventually increase hard drive capacity. Tohoku University |
▶ Database Jiu Jitsu: How ScyllaDB Open Sourced a DynamoDB-Compatible API — Corey Quinn (who you might know from Last Week in AWS and his good humored Twitter snark) joined ScyllaDB’s CEO to discuss vendor lock-in, the economics of open source, and what ScyllaDB is up to. ScyllaDB |
▶ Breakdown Data Silos: Why S3 is the Best Place To Build Your Data Lake — From AWS’s recent AWS Storage Day. It’s basically a 20-minute pitch but ties together a variety of ideas and may be useful if you’re using or considering S3 for anything beyond mere file storage. Matt Sidley (AWS) |
🔨 Code and Tools |
Another Redis Desktop Manager 1.4.0 — An open source desktop client for working with Redis that promises it ‘won’t crash’ if dealing with large numbers of keys. Supports TLS, dark mode, and more – seems like the author has put in some serious effort here. qii404 |
simple-graph: A Simple Graph Database on Top of SQLite — Thanks to its recursive CTE support, you can rig up a graph database on SQLite anyway, but if you’re a Pythonista you might find this abstraction interesting. Denis Papathanasiou |
CYBERTEC PostgreSQL Enterprise Edition: Fully Encrypted & High-Performing — Check out the highly secure PostgreSQL distribution with advanced features like user-friendly monitoring and 24/7 support. CYBERTEC sponsor |
rqlite 5.8: A Distributed Relational Database Built on SQLite — I sometimes wonder if Postgres and SQLite are in a secret fight to have the most number of other databases built on top of them :-) 5.8.0 of this popular distributed database adds support for TLS connections. rqlite |
Database Lab Engine 2.1: Instant Clones of Large Postgres Databases for Dev Environments — Provision independent non-production environments with multi-terabyte Postgres databases in a few seconds without extra costs. Postgres.ai |
💼 Job |
DevOps Engineer at X-Team (Remote) — Join the most energizing community for developers and work on projects for Riot Games, FOX, Sony, Coinbase, and more. X-Team |