Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
| learn:bigdata:hdfs [2014/08/07 13:01] – yehuda | learn:bigdata:hdfs [2022/01/03 16:03] (current) – external edit 127.0.0.1 | ||
|---|---|---|---|
| Line 3: | Line 3: | ||
| * HDFS knows to handle large file (Ex. 1ptb) and split it to [[Data block|Data blocks]] and distrebute it over [[Cluster]]. | * HDFS knows to handle large file (Ex. 1ptb) and split it to [[Data block|Data blocks]] and distrebute it over [[Cluster]]. | ||
| * HDFS has [[wp> | * HDFS has [[wp> | ||
| + | * HDFS Master / Slaves architecture | ||
| + | |||
| + | ===== HDFS Features ===== | ||
| + | * Rack awarness - What happends if entire rack is lost ? | ||
| + | * Reliable storage | ||
| + | * High throughput | ||
| + | ===== Tools ===== | ||
| + | * [[dfsadmin]] | ||
| + | * [[fs shell]] | ||
| + | * Web-UIs | ||
| + | |||
| + | ===== Nodes ===== | ||
| + | ==== Data world ==== | ||
| + | * [[Name node]] - FS Ops, Block Mapping | ||
| + | * [[Secondary name node]] - Checkpoint Ops | ||
| + | * [[Data node]] - Block Ops, Replications | ||
| + | ==== MapReduce world ==== | ||
| + | See: [[MapReduce]] | ||
| + | * Master: [[Job tracker]] - controller of [[Task tracker|Task trackers]] | ||
| + | * [[Task tracker]] | ||
