Tag: EC2

Posted on

EC2 VPC VPN Gets an Update with NAT Traversal, Additional Encryption Options, and More

You can use Amazon Virtual Private Cloud to create a logically isolated section of the AWS Cloud. Within the VPC, you can define your desired IP address range, create subnets, configure route tables, and so forth. You can also use a network gateway to connect the VPC to your existing on-premises network using a hardware Virtual Private Network (VPN) connection. The VPN running in the AWS Cloud (also known as a VPN gateway or VGW) communicates with a customer gateway (CGW) on your network or in your data center (read about Your Customer Gateway to learn more).

Source: EC2 VPC VPN Update – NAT Traversal, Additional Encryption Options, and More | AWS Official Blog

This raises some interesting possibilities for secure applications that would benefit from the heightened security for the data being moved.

Posted on

AWS EC2 instances and the coming leap second

Each EC2 instance has its own clock and is fully under your control; AWS does not manage instance clocks. An instance clock can be affected by many factors. Depending on these factors, it may implement or skip the leap second. It may also be isolated and not synchronize to an external time system. If you need your EC2 instance clocks to be predictable, you can use NTP to synchronize your clocks to time servers of your choice. For more information about how to synchronize clocks, see the following documentation:

Adding the leap second is currently the standard practice. If you use public time servers, like time servers from ntp.org (the default for Amazon Linux AMIs) or time.windows.com (the default for Amazon Windows AMIs), your instance will see the leap second unless these synchronization services announce a different practice.

Source: Look Before You Leap – The Coming Leap Second and AWS | AWS Official Blog

A reminder to check how (y)our EC2 instances are going to deal with this before it happens at the end of June.

Posted on

Did You Know That Amazon Provides Free AWS Training Videos and Labs?

New to AWS and looking to gain a foundational knowledge about key AWS services? Our “Introduction to AWS” series includes free, on-demand instructional videos and labs that enable you to learn about AWS in 30 minutes or less. Start by watching a short video about an AWS service to learn about key concepts and terminology and see a step-by-step console demonstration. Next, get hands-on practice using the AWS service with a free self-paced training lab.

via AWS Training – Free Online Videos and Labs.

I didn’t, but I do now. These are great for learning about the fundamentals of AWS cloud computing. Topics covered include intros to Simple Storage Service (S3), Elastic Compute Cloud (EC2), Identity and Access Management (IAM), Relational Database Service (RDS), and Elastic Load Balancing. Used along with the AWS Free Tier these represent an excellent for anyone to become more familiar with running servers and systems and getting a better understanding of cloud computing.

Posted on

Amazon Announces General Availability of AWS CLI

We are pleased to announce the General Availability (GA) release of the AWS Command Line Interface, a unified tool to manage your AWS services. With just one tool to download and configure, you can control multiple AWS services from the command line and automate them through scripts. The GA release supports 23 services and includes new file commands for Amazon S3. Using a file system command syntax, you can easily list the contents of online buckets, upload a folder full of files, and synchronize local files with objects stored in Amazon S3.

To get started with the AWS CLI, see the User Guide.

via Announcing AWS Command Line Interface – General Availability.

For all you fans of the command line, now you can get some real work done. Bonus: the AWS-CLI is written in Python and is open source so you can follow along on GitHub

Posted on

Amazon Adds Import/Export of Data From External Storage Devices Shipped to AWS Data Centers

AWS Import/Export accelerates moving large amounts of data into and out of AWS using portable storage devices for transport. AWS transfers your data directly onto and off of storage devices using Amazon’s high-speed internal network and bypassing the Internet. For significant data sets, AWS Import/Export is often faster than Internet transfer and more cost effective than upgrading your connectivity.

via AWS Import/Export.

This is an interesting development for folks who need to move really large amounts of data on or off of the cloud. According to a table in the article the cost of shipping the drive and having Amazon do the transfer at the data center becomes cost effective when dealing with data amounts over 1TB being pushed over a 10Mbps connection. That is a lot of data[1].

The service will accept eSATA, USB2.0, and internal SATA drives and transfer your data to an S3 bucket or an EBS Snapshot.

[1]For reference, a single copy of all the data on CALI’s public servers (lessons, court opinions, podcasts, conference video, as so on) weighs in at just over a 1TB.

Posted on

Is The Great Amazon EBS Failure the Beginning of the End For Disk Abstraction?

The promise of network block storage is wonderful: Take a familiar abstraction (the disk), sprinkle on some magic cloud pixie dust so that it’s completely reliable, available over the same cheap network you’re using for app traffic, map it to any instance in a datacenter regardless of network topology, make it so cheap it’s practically free, and voila, we can have our cake and eat it too! It’s the holy grail many a storage vendor, most of whom with decades experience in storage systems and engineering teams thousands strong have chased for a long, long time. The disk that never dies. The disk that’s not a disk.

The reality, however, is that the disk has never been a great abstraction, and the long history of crappy implementations has meant that many behavioral workarounds have found their way far up the stack. The best case scenario is that a disk device breaks and it’s immediately catastrophic taking your entire operating system with it. Failure modes go downhill from there. Networks have their own set of special failure modes too. When you combine the two, and that disk you depend on is sitting on the far side of the network from where your operating system is, you get a combinatorial explosion of complexity.

Magical Block Store: When Abstractions Fail Us « Joyeur.

Fascinating piece on the perils of disk abstraction. Raises a very good question: Why do we worry about disks at all in the cloud? I wonder how many folks would just be tossing data into the cloud without the comfy metaphor of disk and machine to lean on?

Posted on

Best Description of the Likely Cascading Failure That Took Out EC2

Let’s think of a failure mode here: Network congestion starts making your block storage environment think that it has lost mirrors, you begin to have resilvering happen, you begin to have file systems that don’t even know what they’re actually on start to groan in pain, your systems start thinking that you’ve lost drives so at every level from the infrastructure service all the way to “automated provisioning-burning-in-tossing-out” scripts start ramping up, programs start rebooting instances to fix the “problems” but they boot off of the same block storage environment.

You have a run on the bank. You have panic. Of kernels. Or language VMs. You have a loss of trust so you check and check and check and check but the checking causes more problems.

via On Cascading Failures and Amazon’s Elastic Block Store « Joyeur.

Closing in on 36 hours since this melt down began, Amazon has still not been able to restore all of the EC2 instances and EBS volumes that where knocked offline in the #SkynetMassacre. This article is the best explanation of what most likely happened. And the scary part is that it will happen again. And again.

Sadly, there is not a lot to do but try and build enough redundancy into your systems to survive this sort of thing. But it is likely that building that redundancy is going to bring about another melt down at some point. Guess I’ll just need to keep thinking about how to deal with this sort of thing.

Posted on

Using EC2 to (re)Distribute “Repurposed Virtualities”

They give everyone the power to create their own version of Windows and share it with others. Granted, that’s not the kind of thing too many non-techies, or even techies, wake up in the morning with an overwhelming desire to do. But why not? I’m still getting used to the idea of creating my own versions of Windows, haven’t even released anything yet. But since everything I’m building is open source, there’s no reason someone couldn’t take my package, make some changes, and then redistribute it with their customizations. Trust obviously becomes a pretty important issue here.

via Scripting News: Caprica and repurposed virtualities.

This sounds a lot like what already goes on in the EC2 community around public AMIs. If you take a look at the list of public AMIs (over 4,000 at this point) you’ll see that many are bundles of the OS with one or more application packages. For examlple Drupal and Asterisk AMIs are easy to find. Most the public AMIs are Linux-based, but nearly 300 use Windows as the core OS.
I’ve done this sort of thing building a Linux AMI that started with a base image from RightScale to which I added Apache, MySQL, PHP, Drupal, and more configured to work together. Once I saved the AMI, I had a proto-type server that I could use to quickly scale up our web cluster. I’ve also shared the AMI with colleagues interested in getting started with Drupal.
So, having an AMI that contains Dave’s work is certainly doable and an excellent idea. I, for one, would certainly try it out.