hadoop gyan

Based on this paper

The lesson learnt is:

“As shown in fig. 5, job execution time reached its minimum
when the number of mappers was equal to the number of
CPU cores. As a rule of thumb, therefore, datasets should
be uploaded to HDFS with a block size that results in a total
number of blocks close to the number of available CPU cores.”

Advertisements

Tags:

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: