Archive for February, 2007

EC2 demo video from Amazon.

Setting up and Running Amazon EC2 from Windows. Check it out, very cool.

Especially when you can do this sort of stuff: Mux – Video Transcoding Powered by EC2.

Technorati Tags: ,

Comments off

OtherTricks for Spam from Spamassassin

Will have to trying Fake MX records and some of the other tricks on OtherTricks – Spamassassin Wiki at some point.

Technorati Tags: ,

Comments off

Lighttpd, spawn-php and daemontools

This has been sitting forgotten on the draft queue for a while. I’m not using this setup at the moment, but the information is still useful.

Read the rest of this entry »

Comments off

Active/Active HA for Xen using DRBD

HA Migration (not live) Howto for an Active/Active Xen system using DRBD:

So what does all this produce? Node 1 has N DomUs and so does Node 2. Each set of DomUs is on its own drbd device and each node is primary for one of these devices. When a node fails, heartbeat sets the other node as primary for the affected drbd device, activates the LVM VG and LVs and starts the affected set of DomUs via their custom xendomains script (xd1 or xd2). It works great. I’ve rebooted, pulled the plug, and hit the power button and everything fails over OK. There’s a slight delay of about 90 seconds since it isn’t live migration but my environment can tolerate this.

Technorati Tags: , ,

Comments off

VI3 iSCSI Setup Howto

Quick howto with screen-shots for the setup of a iSCSI initiator in VI3.

Up to this point I have not done extensive testing on the overall performance of my setup. What I do know is it performs more than well enough to run an IIS Web Server, an average load SQL Server, and AD Server, an Exchange Server and a File Server without breaking a sweat on the Linux iSCSI Server resources. In addition, the applications respond incredibly well considering the fact that my “Enterprise SAN” cost me less than $500 total. For development purposes to test VMotion, DRS, and HA, this is DEFINITELY a good solution to take a look at. Some brave people, like myself, may even consider using it for production data. I make sure I have a good solid backup every night.

This is running the following setup:

Virtual Machines Running

  • Windows 2003 Domain Controller – 384MB Memory
  • Windows 2003 SQL Server – 512MB Memory (Scripts running consistent read/write/update load on server)
  • Windows 2003 Exchange 2003 Server – 512MB Memory (10 Mailboxes, 5 with a TON of spam being sent for load)
  • Windows 2003 File Server – 384MB Memory

From the looks of some of the comments this is not totally production ready, although this probably has improved in the last six months. It does point to the way things are going though.

Diskless Processing Units, net (PXE, ISCSI, or ?) booting to a hypervisor and running Software Appliances back-ended to Storage Appliance (brandware or software) Units holding the data.

Some hints here for this right with Netapp equipment.

Technorati Tags: , ,

Comments off

VMware Infrastructure 3 Demo

Bit cheesy but pulling a power plug and watching a machine fail over automatically is always going to be a good demo.

Technorati Tags: ,

Comments off

ATA over Ethernet

AoE [1], [2] is a recent protocol developed by Coraid.

ATA over Ethernet is a network protocol registered with the IEEE as Ethernet protocol 0x88a2. AoE is low level, much simpler than TCP/IP or even IP. TCP/IP and IP are necessary for the reliable transmission of data over the Internet, but the computer has to work harder to handle the complexity they introduce.

Users of iSCSI have noticed this issue with TCP/IP. iSCSI is a way to send I/O over TCP/IP, so that inexpensive Ethernet equipment may be used instead of Fibre Channel equipment. Many iSCSI users have started buying TCP offload engines (TOE). These TOE cards are expensive, but they remove the burden of doing TCP/IP from the machines using iSCSI.

An alternative to iSCSI, the AoE specification is 8 pages compared with iSCSI’s 257 pages.

The storage hardware sold by Coraid is very cost effective. The basic 1U four SATA disk chasis SR420 is 2000USD, and with the addition of four 750b SATA disks (say about $450) provides 3Tb raw for under 4000 USD in a 1U with RAID 0,1,5,10 or JBOD. This can be combined with their NAS Gateway the CLN20, to provide a reasonable local network storage system.

With Hitachi’s forthcoming 1TB disk, you get 4TB raw. Crazy!

Of course SATA is not the best for database system but if you have a low access archive type system, or a video silo then this is going to work well.

Technorati Tags: , ,

Comments off

DRBD

I’ve been looking at various storage options recently. DRBD is pretty cool:

And, of course, we only will resynchronize those parts of the device that actually have been changed. DRBD has always done intelligent resynchronization when possible. Starting with the DBRD-0.7 series, you can define an “active set” of a certain size. This makes it possible to have a total resync time of 1–3 min, regardless of device size (currently up to 4TB), even after a hard crash of an active node.

DRBD is a network block level intelligent replication protocol. It syncs local disk blocks on two separate nodes and smartly manages dirty blocks while out of communication. Normally this is used for Active/Passive or Active/Active cluster HA. It could also provide a method for off site replication.

Run a storage server with iSCSI or AoE targets on LVM devices over the DRDb block device, with the secondary node on a remote server via a vpn. Specific details about this setup are discussed in Disaster Recovery with “Tele-DRBD” on the Linux HA Wiki. Some details on LVM over DRBD. Also worth checking the NFS on DRBD page.

Technorati Tags: , , ,

Comments off

Ubuntu and VMware – Losing your ethernet device when migrating

VMWare bases the MAC address of interface on it’s internal UUID. If you shift a machine (copy, rename) vmware asks to update the UUID. You might get something like this:

# ifup eth0
SIOCSIFADDR: No such device
eth0: ERROR while getting interface flags: No such device
SIOCSIFADDR: No such device
SIOCSIFADDR: No such device
eth0: ERROR while getting interface flags: No such device
eth0: ERROR while getting interface flags: No such device

The solution found by hbraga is to check /etc/iftab. Ubuntu on install adds the MAC address for each interface to this file. Either comment out the lines or update them to reflect the correct details.

Technorati Tags: , ,

Comments (33)

The Disk side of the Server game from AnandTech

AnandTech: Server Guide part 2

Even the so called “Nearline” (Seagate) or “Raid Edition” (RE, Western Digital) SATA drives which are made to operate in enterprise storage racks, and which are more reliable than desktop disks, are not made for the mission critical, random transactional applications. Their MTBF (Mean Time Between Failure) is still at least 20% lower than typical enterprise disks, and they will show the similar failure rates when used with highly random server workloads as desktop drives.Also, the current SATA drives on average experience an Unrecoverable Error every 12.5 terabytes written or read (EUR of 1 in 1014 bits). Thanks to the sophisticated drive electronics, SAS/SCSI disks experience these kinds of errors 100 (!) times less. It would seem that EUR numbers are so small that they are completely negligible, but consider the situation where one of your hard drives fails in a RAID-5 or 6 configuration. Rebuilding a RAID-5 array with five 200 GB SATA drives results in reading 0.8 terabytes and writing 0.2 terabytes, in total 1 terabytes. So you have 1/12.5 or 8% chance of getting an EUR on this SATA array. If we look at a similar SCSI enterprise array, we would get a 0.08% chance on one unrecoverable error. It is clear an 8% chance of getting data loss is a pretty bad gamble for a mission critical application.

Another good point that Seagate made in the same study concerns vibration. When a lot of disk spindles and actuators are performing a lot of very random I/O operations in a big storage rack, quite a bit of rotational vibration is the result. In the best case the actuator will have to take a bit more time to get to the right sector (higher seek time) but in the worst case the read operation has to be retried. This can only be detected by the software driver, which means that the performance of the disk will be very low. Enterprise disks can take about 50% more vibration than SATA desktop drives before 50% higher seek times kill the random disk performance.

Technorati Tags: ,

Comments off