Analytics Overhaul, Smarter Crawling, and a Big Migration Week

Week 7

#Silo CDP

  • Full migration to React Router 7 with comprehensive type-safety improvements
  • Analytics reporting system overhauled: password-protected reports, outcomes-first template, v2/v3 report generation with enhanced SQL and comparison support
  • Dark mode overhaul with custom typography, cleaner theme system, and consistent component styling
  • CDN paths centralized, dashboard restructured, server-side bundling issues resolved
  • Node.js upgraded to v24 LTS; environment variables split into service-specific modules
  • IP enrichment caching dramatically improved throughput—batch sizes increased to leverage the cache
  • Batch size tuned for cold cache scenarios to prevent overload on startup
  • Database push safety script made interactive; team enrichment made idempotent

#Inspiration Index

  • Four new link sources added (including a design-focused source), with advisory-locked parallel discovery and dry-run testing
  • Round-robin source selection replaced time-based sorting with Fisher-Yates shuffling for more uniform distribution
  • Domain-based rate limiting with run locks and temporary-down propagation
  • Tristate URL liveness tracking with improved dead/temporary detection
  • Per-device recorder submission system with a dedicated tracking table; recorder now skips fully-submitted websites
  • Gemini 2.5 Flash Lite adopted for frame analysis—faster and more cost-efficient
  • Transient 503 errors from the Google Files API now handled gracefully with retries
  • Video quality validation added before MediaConvert processing
  • Timeout protection and retry logic added to the video frame analysis step
  • Debug URLs surfaced in error handling and Step Functions state for easier debugging
  • Technology filtering added to the Discover view for browsing by tech stack

#Autoscroll Recorder

  • Broken page detection—recordings now abort immediately when a broken page is detected
  • Mobile experience improved with device-aware scroll distance
  • GPU task failover enabled across multiple availability zones for better resilience
  • Service token race conditions fixed
  • Lambda runtime upgraded to Node.js 24
  • Cost estimation tooling added and bot detection improved
  • Batch job cancellation for active jobs—a major workflow improvement for admins
  • CloudWatch logs access added for admin users, visible even without a task ARN
  • Job status polling now updates all job fields, not just distributions
  • Admin debugging tools relocated to the job details section for a cleaner UX