Applying Apache Hadoop to NASA’s Big Climate Data


Comments

Post a Comment

Popular posts from this blog

DeprecatedProperties for Apache Hadoop 3.0.0-alpha4

Deprecated Properties The following table lists the configuration property names that are deprecated in this version of Hadoop, and their replacements. Deprecated property name New property name create.empty.dir.if.nonexist mapreduce.jobcontrol.createdir.ifnotexist dfs.access.time.precision dfs.namenode.accesstime.precision dfs.backup.address dfs.namenode.backup.address dfs.backup.http.address dfs.namenode.backup.http-address dfs.balance.bandwidthPerSec dfs.datanode.balance.bandwidthPerSec dfs.block.size dfs.blocksize dfs.data.dir dfs.datanode.data.dir dfs.datanode.max.xcievers dfs.datanode.max.transfer.threads dfs.df.interval fs.df.interval dfs.encryption.key.provider.uri hadoop.security.key.provider.path dfs.federation.nameservice.id dfs.nameservice.id dfs.federation.nameservices dfs.nameservices dfs.http.address dfs.namenode.http-address dfs.https.address dfs.namenode.https-address dfs.https.client.keystore.resource dfs.client.https.keystore.resource dfs.https...
Read more

HADOOP DISTRIBUTED FILE SYSTEM ( HDFS )

FIRST THING IN OUR MIND THAT WHERE TO USE THIS HADOOP   Let us see and consider one use case. Suppose You are the Manager of an Organisation, and you want how many times these words like POWER FAILURE, CSAT NO , REFUND used by his customers in their emails. If we had structured data than it can easily managable without using Hadoop provided Volume warehouse management proportions, but we have Unstructured data  in emails, so we will have to solve this problem using Hadoop. Solutions would be. As we know that Hadoop Components have two things. 1) HDFS ( HADOOP DISTRIBUTED FILE SYSTEM ) for Distributed storage. 2) MAPREDUCE for prarallel data processing. So execution steps. Load the data into HDFS. Analyse the data in Parallel using MAP-REDUCE. Store results back into HDFS. Retrieve results and Visualize. So, We need to know deeply that What is HDFS exactly and how is it processed and stored using its Architecture. Next➱
Read more