Apache Pinot™
Realtime distributed OLAP datastore, designed to answer OLAP queries with low latency
Pinot is proven at scale in LinkedIn powers 50+ user-facing apps and serving 100k+ queries
What is Apache Pinot?
#
FeaturesFast queries
Pinot can filter and aggregate petabyte data sets with P90 latencies in the tens of milliseconds—fast enough to return live results interactively in the UI.
High concurrency
With user-facing applications querying Pinot directly, it can serve hundreds of thousands of concurrent queries per second.
Batch and streaming ingest
Ingest from Apache Kafka, Apache Pulsar, and AWS Kinesis in real time. Batch ingest from Hadoop, Spark, AWS S3, and more. Combine batch and streaming sources into a single table for querying.
Upserts
Ingest the same record many times, but see only the latest value at query time. Upserts are built-in and production-tested since version 0.6.
Versatile joins
Perform arbitrary fact/dimension and fact/fact joins on petabyte data sets.
Rich indexing options
Choose from pluggable indexes including timestamp, inverted, star-tree, Bloom filter, range, text, JSON, and geospatial options.
Built for scale
Pinot is horizontally scalable and fault-tolerant, adaptable to workloads across the storage and throughput spectrum.
SQL query interface
The highly standard SQL query interface is accessible through a built-in query editor and a REST API.
Built-in multitenancy
Manage and secure data in isolated logical namespaces for cloud-friendly resource management.
#
Ingest and Query Options#
User-Facing AnalyticsBuilding Latency Sensitive User Facing Analytics via Apache Pinot
Using Apache Kafka and Apache Pinot for User-Facing Analytics
#
Installs EverywhereInstall:
- Using Helm
- Using Binary
- Build From Source
helm repo add pinot https://raw.githubusercontent.com/apache/pinot/master/helmkubectl create ns pinothelm install pinot pinot/pinot -n pinot --set cluster.name=pinot
VERSION=1.1.0wget https://downloads.apache.org/pinot/apache-pinot-$VERSION/apache-pinot-$VERSION-bin.tar.gztar vxf apache-pinot-*-bin.tar.gzcd apache-pinot-*-binbin/quick-start-batch.sh
# Clone a repogit clone https://github.com/apache/pinot.gitcd pinot
# Build Pinotmvn clean install -DskipTests -Pbin-dist
# Run the Quick Democd pinot-distribution/target/apache-pinot-*-SNAPSHOT-bin/apache-pinot-*-SNAPSHOT-binbin/quick-start-batch.sh