官术网_书友最值得收藏!

Saving money using Amazon EC2 Spot Instances to execute EMR job flows

Amazon EC2 Spot Instances allow us to purchase underutilized EC2 compute resources at a significant discount. The prices of Spot Instances change depending on the demand. We can submit bids for the Spot Instances and we receive the requested compute instances if our bid exceeds the current Spot Instance price. Amazon bills these instances based on the actual Spot Instance price, which can be lower than your bid. Amazon will terminate your instances, if the Spot Instance price exceeds your bid. However, Amazon does not charge for partial Spot Instance hours if Amazon terminated your instances. You can find more information on Amazon EC2 Spot Instances at http://aws.amazon.com/ec2/spot-instances/.

Amazon EMR supports using Spot Instances both as master as well as worker compute instances. Spot Instances are ideal to execute nontime critical computations such as batch jobs.

How to do it...

The following steps show you how to use Amazon EC2 Spot Instances with Amazon Elastic MapReduce to execute the WordCount MapReduce application:

  1. Follow steps 1 to 9 of the Running Hadoop MapReduce v2 computations using Amazon Elastic MapReduce recipe.
  2. Configure your EMR cluster to use Spot Instances in the Hardware Configuration section. (Refer to step 10 of the Running Hadoop MapReduce v2 computations using Amazon Elastic MapReduce recipe). In the Hardware Configuration section, select the Request Spot checkboxes next to the instance types.
  3. Specify your bid price in the Bid price textboxes. You can find the Spot Instance pricing history in the Spot Requests window of the Amazon EC2 console (https://console.aws.amazon.com/ec2).
    How to do it...
  4. Follow steps 11 to 16 of the Running Hadoop MapReduce v2 computations using Amazon Elastic MapReduce recipe.

There's more...

You can also run the EMR computations on a combination of traditional EC2 on-demand instances and EC2 Spot Instances, safe guarding your computation against possible Spot Instance terminations.

Since Amazon bills the Spot Instances using the current Spot price irrespective of your bid price, it is a good practice to not set the Spot Instance price too low to avoid the risk of frequent terminations.

See also

The Running Hadoop MapReduce v2 computations using Amazon Elastic MapReduce recipe.

主站蜘蛛池模板: 甘德县| 洪泽县| 衢州市| 九龙坡区| 呼和浩特市| 平安县| 玛多县| 江门市| 襄樊市| 新沂市| 乌鲁木齐县| 扎鲁特旗| 武强县| 元江| 汾西县| 迁西县| 五华县| 汝南县| 田林县| 满洲里市| 贡嘎县| 沙田区| 渝中区| 军事| 德保县| 甘洛县| 清水县| 天气| 梧州市| 磐安县| 拉萨市| 白沙| 永胜县| 宾川县| 喀什市| 上蔡县| 精河县| 江达县| 滁州市| 赣州市| 石泉县|