Hadoop

Hadoop is framework that has set of tools to distrebute and proccess data over clasters.

• Scalable
• Flexible
• Fault-tolerant
• Intelligent

Main tools are HDFS (HaDoop File System), MapReduce and YARN

Single node Cluser

• Standalon mode - all hadoop components run under single JVM
• Pesodo Destributed - each deamon runs under seperated JVM
• Fully Destributed - each deamon runs under seperated maching

see Install Hadoop eco system (single mode)

see more at http://incubator.apache.org/

Hive, Pig

Hbase, Cassandra

HCatalog, Lucene, Hama, Crunch

Avro, Thrift

Drill, Mahout

Sqoop, Flume, Chukwa

Ambari, Zookeeper, Oozie

HDT, Konx, Spark

• New-York Times - Want to convert 4 TB of articales to PDF. thay did it with AWS less then 24 hours and it cost them about $240!