DuckDB v1.5.0 introduces major advancements, including a reworked CLI and the elevation of GEOMETRY to a native core type
TLDR Data 2026-03-12
Deep Dives
Driving data enhancement & recruitment success with LinkedIn’s unified integrations (14 minute read)
Using a declarative transformation layer, Temporal for orchestration, Kafka for streaming, Espresso for persistence, and support for both partner-push (BuildIn) and LinkedIn-pull/push (BuildOut) models with idempotent, secure, observable flows, LinkedIn cut partner onboarding time by 72%, expanded data coverage 4x, boosted data completeness, and provided a stable, governed foundation for features like LinkedIn’s Hiring Assistant.
Building High Throughput Payment Account Processing (9 minute read)
To handle extreme “hot-key” traffic on high-activity payment accounts which previously hit sequential processing limits of 3–4 ops/sec per account, causing multi-hour processing times for bulk jobs, Uber uses a three-service architecture to batch financial updates into ~250 ms windows (using Redis for coordination/queuing) with an atomic write per batch, and offloads immutable audit logs asynchronously.
Opinions & Advice
Your Pipeline Succeeded. Your Data Didn’t (12 minute read)
A BigQuery-native anomaly detection system can catch silent data loss by monitoring ingestion volume across hundreds of tables using only built-in features like INFORMATION_SCHEMA logs and AI.DETECT_ANOMALIES. Implemented as a single dbt model, it detects abnormal drops in data volume without external tooling or per-table rules, helping teams identify partial pipeline failures that traditional pipeline success checks miss.
How I improved my analytics agent reliability from 45% to 86% (15 minute read)
Analyzing failures in an analytics agent and fixing issues in tests, date-selection rules, documentation, and especially the underlying data model improved reliability from 45% to 86%. The results show that most gains in context engineering come from clearer data models, explicit rules, and better documentation rather than complex agent architectures.
Data Contracts Won’t Save You If Your AI Agent Can’t Read Them (8 minute read)
Traditional data contracts and governance policies, written for human oversight, are largely invisible to AI-powered machine consumers, exposing organizations to undetected errors and policy breaches. Contract controls like freshness SLAs, usage restrictions, quality thresholds, and semantic definitions are typically stored in config files and documentation, making them inaccessible to autonomous agents at query time. Critical governance elements must transition from human-readable specifications to machine-readable, queryable metadata and enforceable runtime policies.
Silent Data Loss in ClickHouse: 3 Reasons Your Distributed Queue Keeps Growing (7 minute read)
Inserts into ClickHouse Distributed tables seem to succeed instantly but silently pile up as .bin files in an on-disk queue that never reaches the underlying ReplicatedMergeTree tables, causing permanent data loss and missing rows. The root causes include ClickHouse Keeper downtime putting replicas into read-only mode and stalling background flushes, as well as oversized insert blocks killing the flush and blocking the entire queue.
Launches & Tools
Announcing DuckDB 1.5.0 (16 minute read)
DuckDB v1.5.0 introduces major advancements, including a reworked CLI, support for the VARIANT type (with binary storage for enhanced compression and query performance), and the elevation of GEOMETRY to a native core type with automatic column shredding—reducing storage by up to 3x. Notable concurrency and aggregate optimizations yield a 17% TPC-H SF100 throughput boost and up to 40% faster aggregates.
Scaling with Airflow 3.2: When to Defer and When to Use Native Async Operators (8 minute read)
Apache Airflow 3.2 adds native async support to PythonOperator, allowing high-throughput I/O workloads to run concurrently inside a single worker without triggerer overhead. Combined with Dynamic Task Iteration, this approach dramatically improves performance for micro-batch tasks like API calls or SFTP transfers.
Orchestrate dbt Core in Production with Kestra (5 minute read)
Kestra orchestrates dbt Core transformations by integrating scheduling, dependency management, retries, and alerting within a declarative YAML-based workflow. Its 1,200+ plugin ecosystem enables end-to-end orchestration across ingestion, transformation, and activation layers, while providing full cross-stack lineage tracking with Assets.
Miscellaneous
Tenable Research uncovered nine critical cross-tenant vulnerabilities, dubbed “LeakyLooker,” in Google Looker Studio, exposing organizations’ data across BigQuery, Sheets, and other GCP connectors to exfiltration and manipulation. These flaws enabled attackers to exploit credential handling and SQL injection vectors to access, modify, or steal data with zero or one click, breaking established BI platform trust boundaries. Google has remediated all vulnerabilities.
Top AI GitHub Repositories in 2026 (10 minute read)
Most impactful open-source AI projects include OpenClaw (210k+ stars, fastest-growing ever local AI assistant with 50+ integrations and self-extending skills), Ollama (local LLM runtime that enables privacy-focused on-device models), n8n, Langflow, Dify, LangChain, and more, reflecting 2026 trends toward local/privacy-first AI and agentic autonomy.
Quick Links
A Google Trends style view of the GitHub ecosystem, powered by ClickHouse.
Your Data Agents Need Context (7 minute read)
Data agents are only useful when grounded in a well-maintained context layer that captures business definitions, source of truth data, governance rules, and tribal knowledge so they can answer real questions reliably.