Design Data-Intensive Applications [Read More]

Design Data-Intensive Applications [Read More]

Design Data-Intensive Applications [Read More]

Hive Basics ACID vs non-ACID tables, non-ACID table is preferred (ACID tables also need compaction, combine delta folder with base folder, cannot be read properly by Spark) Choose proper format (compression) for tables Enable storage index, ‘orc.create.index’=’true’ Bucketing Bloom filter Partition, avoid too many partitions Record insertion should use sort... [Read More]

Big Data SQL Hive Apache Phoenix on HBase SparkSQL Presto IBM BigSQL HBase http://www.uml.org.cn/bigdata/201804131.asp NiFi https://dzone.com/articles/apache-spark-and-apache-nifi-integration-part-2-of Knox Gateway [Read More]