Archive for Solaris

Sun and IBM

I haven’t seen much yet to confirm this deal, but if it does happen next week it mark an interesting change in course.  The thing that would interest me greatly would be zfs on linux – with Sun owned by IBM and their investment in Linux it might lead to a reconcilation between Sun and Linux licenses.  While Solaris has come a long way in the last couple years, but the only reason why I use solaris is zfs.  Linux is still a much more flexible, effective and deployable solution. 


Reblog this post [with Zemanta]

Comments off

Fishworks – Hybrid Storage

Some more detail on Sun’s new storage platform, Fishworks.

Hybrid Storage Pools in the 7410:


The write performance of 7200 RPM drive isn’t terrific. The appalling thing is that the next best solution — 15K RPM drives — aren’t really that much better: a factor of two or three at best. To blow the doors off, the Sun Storage 7410 allows up to four write-optimized flash drives per JBOD each of which is capable of handling 10,000 writes per second. We call this flash device Logzilla.

Logzilla is a flash-based SSD that contains a pretty big DRAM cache backed by a supercapacitor so that the cache can effectively be treated as nonvolatile. We use Logzilla as a ZFS intent log device so that synchronous writes are directed to Logzilla and clients incur only that 100μs latency. This may sound a lot like how NVRAM is used to accelerate storage devices, and it is, but there are some important advantages of Logzilla. The first is capacity: most NVRAM maxes out at 4GB. That might seem like enough, but I’ve talked to enough customers to realize that it really isn’t and that performance cliff is an awful long way down. Logzilla is an 18GB device which is big enough to hold the necessary data while ZFS syncs it out to disk even running full tilt. The second problem with NVRAM scalability: once you’ve stretched your NVRAM to its limit there’s not much you can do. If your system supports it (and most don’t) you can add another PCI card, but those slots tend to be valuable resources for NICs and HBAs, and even then there’s necessarily a pretty small number to which you could conceivably scale. Logzilla is an SSD sitting in a SAS JBOD so it’s easy to plug more devices into ZFS and use them as a growing pool of intent log devices.


The standard practice in storage systems is to use the available DRAM as a read cache for data that is likely to be frequently accessed, and the 7000 Series does the same. In fact, it can do quite a better job of it because, unlike most storage systems which stop at 64GB of cache, the 7410 has up to 256GB of DRAM to use as a read cache. As I mentioned before, that’s still not going to be enough to cache the entire working set for a lot of use cases. This is where we at Fishworks came up with the innovative solution of using flash as a massive read cache. The 7410 can accomodate up to six 100GB, read-optimized, flash SSDs; accordingly, we call this device Readzilla.

With Readzilla, a maximum 7410 configuration can have 256GB of DRAM providing sub-μs latency to cached data and 600GB worth of Readzilla servicing read requests in around 50-100μs. Forgive me for stating the obvious: that’s 856GB of cache —. That may not suffice to cache all workloads, but it’s certainly getting there. As with Logzilla, a wonderful property of Readzilla is its scalability. You can change the number of Readzilla devices to match your workload. Further, you can choose the right combination of DRAM and Readzilla to provide the requisite service times with the appopriate cost and power use. Readzilla is cheaper and less power-hungry than DRAM so applications that don’t need the blazing speed of DRAM can prefer the more economical flash cache. It’s a flexible solution that can be adapted to specific needs.

Some back story, a GUI screenshot and detail on Dtrace with:Fishworks: Now it can be told.  And some detail on topology.

I wonder if it will be possible to get the Log and Read cache devices seperately.

Comments off

New Sun Storage Systems

Sun have come out with some storage appliances. The have some compelling functionality, like being able to trace critical path performance.

So there are many good reasons why the 7000 series is cool – the integrated flash devices, the hardware itself, blah, blah blah. Here’s the amazing part – the hardware isn’t even the coolest feature. It’s the software. The ability to *in real time* drop in new tracing events to see what’s really happening on the device is just unbelievable.

How many times have you seen your NAS devices suddenly “go slow”? And you have *no clue* as to why. I can tell you it happens often when running big infrastructure. You dig around for a while and maybe you can figure out that it’s one machine and if you’re particularly good you can figure out one user on one machine and slap their hands. With the 7000 you get the ability in real time to dig into whats being done using which protocol by user, by file, but whatever you want. It’s stunning to see, and incredibly useful in managing the infrastructure. For a mostly detailed overview of the capabilities, check out Bryan Cantrill’s presentation on analytics.

The mid and high end models both support ssd flash to speed up I/O.

The SSDs are used explicitly for caching and logging, and only the 7410 offers both — the 7210 has read-biased SSDs, and the 7110 doesn’t have SSD support. In discussions with Sun engineers, they claimed that the addition of the read-biased SSD caching in conjunction with ZFS’ predictive caching algorithms means that 7200RPM SATA drives perform just as well, if not better than 10K SAS drives. In fact, they’re conducting trials to determine if they can use 4200RPM SATA drives in these devices without sacrificing I/O performance. If that’s possible, then the price point, power consumption, and heat generation drops across the board.

I’ll be looking for reviews with interest to see how these compare with similar NAS systems.

From smugmug.

Comments off


Some numbers on nocacheflush with 2530 Array and ZFS.

Comments off

Is ZFS ready for primetime?

This post on Is ZFS ready for primetime? is disconcerting. I’ve put a lot of effort recently in engineering a zfs based storage server to backend a xen cluster. I haven’t put much up here and a back injury has restricted the amount of seat time. Without much tuning I’ve put together a TB+ system that provides 85Mb/s over NFS to a RAIDZ2 pool.

Throughout the whole period it has been a constant struggle to get Opensolaris working, figure out ancient Solaris ideas about system management. Sort out a kernel bug that caused crashes when running bonnie++.

Being held ransom by the idea of needing a Sun support contract at some future unknown date due to a bug in zfs seems to me as a losing position. Being forced into that position I think misses the point of opensource. Zfs provides striking advantages with checksums and simple snapshots, easy offline remote replication. But without the idea that your data is safe there is no point.

I can put together a Linux storage system without thinking and manage it without much thought. The lack of constant time snapshots is going to hurt, but at least I know the exact same system will still work in 2 years without any changes.

Comments (2)

Nexenta Flash Demos

I set of flash demos look at Nexenta Operating System. Particular interest at creating zones and using BrandZ for etch. The demos seemed to be based on the new NexentaCP server platform which is the basis for NexentaStor.

Comments off

CIFS in Solaris

A clear look at CIFS in Solaris by Alan Wright:

Many people assumed or desired that the Solaris CIFS server project would be like Samba but what would that achieve? Sure, it would avoid breaking any eggs, i.e. avoid making substantial changes to Solaris, but Samba is available on Solaris today. There is no point in creating another Samba. If you truly want an integrated CIFS implementation, that can really inter-operate with Windows at a fundamental level, the operating system has to support certain core features. Eggs will have to be broken.

Comments off

Nexenta Storage Appliance

Nexenta Systems the guys behind the Debianised version of OpenSolarias have release a commercial storage appliance version of Nexenta:

Nexenta Storage Appliance is designed and built to operate as 2nd tier storage alongside pre-existing commercial storage, providing online continuation of data for months and years, with tapes relegated to archival purposes only. The appliance is targeted for 2nd-tier NAS and iSCSI applications requiring extremely low cost storage as well as dramatically simplified provisioning, expansion, backup, replication and archiving. NexentaStor can also be used as a primary NAS in businesses that wish to expand at closer to commodity pricing.

They have a VMware based evaluation version for download, and access to the bare metal install requires talking to their sales team.

From Martin Man.

Comments off

Nexenta Core Platform

Nexenta is a Gnu-land based, Ubuntu derived distribution with a Solaris kernel. It does lag the mainline OpenSolaris for features and bug fixes a little, but is easy to install and brings the apt-get goodness to Solaris. The have announce a new platform called: NexentaCP (Nexenta Core Platform).

NexentaCP is Dapper/LTS-based core Operating System Platform distributed as a single-CD ISO, integrates Installer/ON/NWS/Debian and provides basis for Network-type installations via main or third-party APTs (NEW).

First “unstable” b65-based ISO with ZFS/Boot-capable installer available
as usual at:

Please give it a try and start building your own APT repositories and communities today!

Note: this version of installer supports ZFS/Boot type of installations on single disk or 2 mirror configuration. For now, only “Auto” partitioning mode could be used for ZFS root partition creation.

More details on NexentaCP will be available soon.

Comments off

MySQL backups with ZFS

This tech tip for “Consistent MySQL backups using ZFS snapshots“, is one of the compelling things about zfs. Maybe Netapp is better at the high level for this stuff with systems like Oracle, but for the other 90% of us zfs really creates potential options.

Comments off