HDFS Architecture
HDFS Architecture
Popular posts from this blog
DeprecatedProperties for Apache Hadoop 3.0.0-alpha4
Deprecated Properties The following table lists the configuration property names that are deprecated in this version of Hadoop, and their replacements. Deprecated property name New property name create.empty.dir.if.nonexist mapreduce.jobcontrol.createdir.ifnotexist dfs.access.time.precision dfs.namenode.accesstime.precision dfs.backup.address dfs.namenode.backup.address dfs.backup.http.address dfs.namenode.backup.http-address dfs.balance.bandwidthPerSec dfs.datanode.balance.bandwidthPerSec dfs.block.size dfs.blocksize dfs.data.dir dfs.datanode.data.dir dfs.datanode.max.xcievers dfs.datanode.max.transfer.threads dfs.df.interval fs.df.interval dfs.encryption.key.provider.uri hadoop.security.key.provider.path dfs.federation.nameservice.id dfs.nameservice.id dfs.federation.nameservices dfs.nameservices dfs.http.address dfs.namenode.http-address dfs.https.address dfs.namenode.https-address dfs.https.client.keystore.resource dfs.client.https.keystore.resource dfs.https...
HADOOP DISTRIBUTED FILE SYSTEM ( HDFS )
FIRST THING IN OUR MIND THAT WHERE TO USE THIS HADOOP Let us see and consider one use case. Suppose You are the Manager of an Organisation, and you want how many times these words like POWER FAILURE, CSAT NO , REFUND used by his customers in their emails. If we had structured data than it can easily managable without using Hadoop provided Volume warehouse management proportions, but we have Unstructured data in emails, so we will have to solve this problem using Hadoop. Solutions would be. As we know that Hadoop Components have two things. 1) HDFS ( HADOOP DISTRIBUTED FILE SYSTEM ) for Distributed storage. 2) MAPREDUCE for prarallel data processing. So execution steps. Load the data into HDFS. Analyse the data in Parallel using MAP-REDUCE. Store results back into HDFS. Retrieve results and Visualize. So, We need to know deeply that What is HDFS exactly and how is it processed and stored using its Architecture. Next➱
Comments
Post a Comment