What’s next in data storage – beyond RAID drives….

Data is exploding, growing 10X every five years. In 2008, it was projected that over 800 Exabytes (one million terabytes) of digital content existed in the world and that by 2020 that number is projected to grow over 35,000 Exabytes. What’s fueling the growth? Unstructured digital content. Over 90% of all new data created in the next five years will be unstructured digital content, namely video, audio and image objects. The data storage, archive and backup of large numbers of digital content objects is quickly creating demands for multi-petabyte (one thousand terabytes) storage system.

What is one of the suggested solutions? Information Dispersal. First, a little background.

Current data storage systems based on RAID arrays were not designed to scale to this type of data growth. As a result, the cost of RAID-based storage systems increases as the total amount of data storage increases, while data protection degrades, resulting in permanent digital asset loss. With the capacity of storage devices today, RAID-based systems cannot protect data from loss. Most IT organizations using RAID for big data storage incur additional costs to copy their data two or three times to protect it from inevitable data loss.

RAID schemes are based on parity, and at its root, if more than two drives fail simultaneously, data is not recoverable. The statistical likelihood of multiple drive failures has not been an issue in the past. However, as drive capacities continue to grow beyond the terabyte range and storage systems continue to grow to hundreds of terabytes and petabytes, the likelihood of multiple drive failures is now a reality.

Further, drives aren’t perfect, and typical SATA drives have a published bit rate error (BRE) of 1014, meaning that once every 100,000,000,000,000 bits, there will be a bit that is unrecoverable. Doesn’t seem significant? In today’s big data storage systems, it is.

The likelihood of having one drive fail, and encountering a bit rate error when rebuilding from the remaining RAID set is highly probable in real world scenarios. To put this into perspective, when reading 10 terabytes, the probability of an unreadable bit is likely (56%), and when reading 100 terabytes, it is nearly certain (99.97%).

RAID advocates will tout its data protection capabilities based on models using vendor specified Mean Time To Failure (MTTF) values. In reality, drive failures within a batch of disks are strongly correlated over time, meaning if a disk has failed in a batch, there is a significant probability of a second failure of another disk.

Having just experienced a dual tetrabyte RAID drive failure on our media drive, I completely understand the concerns. Now we are looking for an expanded backup solution. While this is not yet ready for smaller operations, it will be available soon.

Information Dispersal, a new approach for the challenges brought on by big data, is cost-effective at the petabyte and beyond levels for digital content storage. Further, it provides extraordinary data protection, meaning digital assets are preserved essentially forever.

Information Dispersal Basics – Information Dispersal Algorithms (IDAs) separate data into unrecognizable slices of information, which are then distributed—or dispersed—to storage nodes in disparate storage locations. These locations can be situated in the same city, the same region, the same country or around the world.

Each individual slice does not contain enough information to understand the original data.  In the IDA process, the slices are stored with extra bits of data which enables the system to only need a pre-defined subset of the slices from the dispersed storage nodes to fully retrieve all of the data.

Because the data is dispersed across devices, it is resilient against natural disasters or technological failures, like drive failures, system crashes and network failures. Because only a subset of slices is needed to reconstitute the original data, there can be multiple simultaneous failures across a string of hosting devices, servers or networks, and the data can still be accessed in real time.

uppose an organization needs one petabyte of usable storage, and has the requirement for the system to have six nines of reliability – 99.9999%. Here’s how a system built using Information Dispersal would stack up against one built with RAID and Replication.

To meet the reliability target, the Dispersal system would slice the data into 16 slices and store those slices with a few extra bits of information such that only 10 slices would be needed to perfectly recreate the data – meaning, the system could tolerate six simultaneous outages or failures and still provide seamless access to the data. The raw storage would increase by 1.6 (16/10) times the usable storage, totaling 1.6 petabytes.

To meet the reliability target for one petabyte with RAID, the data would be stored using RAID 6, and replicated two, three or even four times possibly using a combination of disk/tape . The raw storage would increase by .33 for the RAID 6 configuration, and then be replicated three times for a raw storage of four times, totaling four petabytes.

Comparing these two solutions side by side for one petabyte of usable storage, Information Dispersal requires 60% the raw storage of RAID 6 and replication on disk, which translates to 60% of the cost.

When comparing the raw storage requirements, it is apparent that both RAID 5 and RAID 6 require more raw storage per terabyte as the amount of data increases. The beauty of Information Dispersal is that as storage increases, the cost per unit of storage doesn’t increase while meeting the same reliability target.

Once vendors start addressing the small to medium storage capacity requirements, this will be a viable alternative to RAID. As of now, this is really only offered to organisations using 500 terabytes or more.

Advertisements

Evernote For Business – A quick look

For those of you using the free Evernote or Premium Evernote, and you collaborate with others, you may want to look at this new version of Evernote – note it is not yet available on all platforms.

A quick background – Evernote is a free, Web-based service that lets users create and organize free-form notes. It has applications for all major desktop and mobile platforms. The company says more than 45 million people use Evernote, to house and organize a variety of unstructured content.Evernote also creates Web Clippers for all of the major desktop browsers, so users can send Web pages directly into Evernote with a mouse click. Each user gets an Evernote email address, so you can send emails directly into Evernote as well. Evernote supports documents, lets users create voice notes and even tags entries with location.   Evernote Business – New Capabilities

Evernote Business lets an organization deploy and manage the Evernote application on behalf of employees, extending information discoverability and sharing company-wide.

The software includes Business Notebooks, collections of Evernote entries along topical lines, which can now be shared with co-workers; and the Business Library, which includes Business Notebooks and centralized administrative and company communications.

Evernote Business also adds Related Notes to the user interface. This feature digs into a company’s Evernote trove, exposing information in a contextual way, depending on what the user is working on.   Administrators create the Business Library, which centralizes select information from all company users. Sharing and collaborating is much more natural now.

Every time you interact with Evernote, the program takes every opportunity to show you relevant notes. For example, when you search inside Evernote, it lists your notes and those shared with co-workers or stored in the Business Library.

But it also works when you’re not searching. These Related Notes come from any Notebook the user has joined. With Related Notes, when you start creating a note, it searches for related content, not just in your own notes, but across the business.

In many ways, these new capabilities start to unlock the potential of an application like Evernote. Although the Evernote Business isn’t marketed as a wiki platform for SMBs, it’s starting to feel like one, at least for those who think of wikis as a way to share and discover knowledge. What makes Evernote enticing here is that it’s more of a serendipitous discovery than a forced organization of information.

Here’s a snapshot (from their website) of screens you’ll see with the new version –

evernote1

With wiki software users must explicitly use them — that is, launch them, login and enter data in a company-defined scheme — when they want to share information, whereas with Evernote that sharing happens as part of the experience. For example, if you’ve created an Evernote entry about a recent business trip related to a project, when another user creates an Evernote entry about that project, those entries are automatically linked.

Evernote Business, priced at $10 per user per month, includes a Web-based administrative application that can be called from the Evernote desktop app. Employees already using the free version of Evernote get upgraded to a more premium version automatically. Evernote has also beefed up its support for business customers, who now get to talk live with a support person.

Evernote Business with just the basic features will ship on all major platforms (Mac, iOS, Android, Windows desktop) starting Tuesday. The serendipitous discovery feature will initially be available only on the Mac, then on the iPhone, iPad, Android and Windows next year.

A Web Clipper capability (described below), which brings Evernote Notes to a Google search, will be available only on Google Chrome at launch, but it will soon be available on Firefox, followed by Safari and Internet Explorer.

Fix Troublesome Wireless Connections by Removing their Profile

It can happen that you start having problems with a wireless network which worked well in the past. This might be due to the fact that the its settings may have been changed accidentally or its network profile, as saved on your computer, got corrupted for some reason. In such scenarios it helps to delete the profile of your wireless network and start fresh: have Windows 7 detect it again, introduce your connection details, etc. This tutorial will show how to delete the network profile of a troublesome wireless connection.

Removing the Troublesome Network Profile   Open the Network and Sharing Center. There, on the left side column, click on “Manage wireless networks”.

networks

In the Manage Wireless Networks window, you can see the profiles of all the wireless networks to which you connected to in the past.

Select the network with which you are having trouble, and click Remove.

Networks2

Confirm that you want to remove the network profile.

networks3

The profile of the wireless network is now deleted. Windows will detect that network as if it was a new discovery and you will be able to enter all the details again and connect to it.

ITunes 11 – First Look

First, we’d like to say we had a very busy November 2012 and subsequently did not post many blogs these last couple weeks. This does not mean we have not been working hard to present you with additiional information – we have (gulp!) plenty to say. Look for a robust posting of information these next couple of weeks. Now on to our post . . .

Whatever you think of Apple’s venerable iTunes software — and even folks who don’t always love it certainly spent a lot of the time in the joint — you probably agree it was due for a refresh.

Apple’s remodeling job arrived today, somewhat later than expected. As promised, iTunes 11 brings a cleaner, more modern look that makes it simpler to search, browse, and create and modify playlists.

Features are more closely tied to Apple’s iCloud online service. Assuming you have an Internet connection, you can play any of your music, movies and TV shows right from iCloud. You can obviously download the media to sync to a device or play when you’re offline. iCloud also remembers your place in a movie or TV show so you can resume watching from where you left off, even if you started viewing it say on an iPhone, iPad or Apple TV and now want to finish on a computer.

The new iTunes features what Apple likes to refer to as an edge-to-edge design that eliminates the familiar source list sidebar, at least from the default view. You can bring it back from one of the menus in iTunes. When it is visible, you can click the + button at the bottom of the screen to create a new playlist. But even if it is out of sight, you can display your playlists just by merely dragging a song in your library.

There’s a new Up Next feature which, as its name suggests, lets you know the next song that will play. From within the list, you can remove or change the order of things. You can summon the feature from a new MiniPlayer as well.

And there’s a hook to get you to part with more of your money in the iTunes store. Even as you’re listening to a song, you can browse related music in the store.

Obviously, I’ll need to spend more time with iTunes 11 to come to a true evaluation of the upgrade. But on the surface it appears that many of the changes will bring music to your ears.