How Storage biz will weather the world financial crisis...

Since the world is now moving to cost-cutting and cost-saving to weather the latest global financial storm. Every companies are trying to see how they can reduce their expenses and even go a great length to even cut employees.

Today, after a nice party, I think of the unavoidable crisis in terms of storage business. In the past Storage Area Network investment is very huge, normally exceeding server/network investment.

The question is how to do more with less?

The famous buzz words now are 'consolidation', 'virtualization', 'etc.'. But does it really work?

If you are going to do consolidation, you need to spend money first before you see the benefit. And most benefits come from human capital investment (less storage, less servers = less people to manage). The other camp may argue that consolidation/virtualization give companies flexibility and scalability. However, it comes with the cost.

Step 1. Assess

So what is the idea? First things first, you need to know if your current infrastructure is sound. Having said that, that means the first step is to conduct an assessment to figure out if your infrastructure is up to what you expect it to be.

If it is, then you can proceed to step 2. If not, you have to decide if you really have to upgrade or make it what you expect then proceed. In most cases, you probably can live with what you have. Otherwise, you would have changed it long time ago.

Step 2. Architect

How should we architect? If you have the SAN, the storage that you need, what you need is to lose what you don't need. That means all the files that you don't open in 2 years, it should be saved outside the precious Storage infrastructure. Things like DVD jukeboxes come into play to move unused files. This could be done inexpensively using opensource products to analyze all your files. So at this stage, you come up with the plan that you want to do.

If you have to invest in new equipment, you may need to look at all alternatives. Then get the management to sign off on it.

Step 3. Actualize

Once you know what to do, you get approvals to do. It's time to do it.

Disclaimer: I am now working part-time as a Principal Consultant to a large multi-national Storage company.

Fiber-Channel, here we come!

Since majority of my posts earlier focused on iSCSI/IP SAN. And I've moved on and worked with a major Fiber-Channel player; if not the largest SAN switch provider. Since joining the company, I've been busy with major league customers like a major Stock Exchange in Asia (top-2) and a major Rail operator, a major Networking equipment provider (top-3), a major insurance company in China, and more.

The reason I write about Fiber-Channel is that now is the time to talk about the emerging technology in the Data Center. Cisco calls it Data Center 3.0, the other calls it Data Center Fabric. Right now, the standard body is forming, there will be a standard for Fiber-Channel over Ethernet. Why? It's simple. TCP/IP is lossy. Fiber-Channel is not. The list goes on and on.

Why companies need to change the equipment to embrace the new technology? It is harder to maintain two separate networks - one Ethernet network for communications, one Storage network for data. iSCSI/IP SAN camp will try to eliminate Fiber-Channel Storage networks asking companies to opt for one interface (Gigabit/10Gigabit) for both data (iSCSI) and communications (LAN or TCP/IP). This FC0E (Fiber Channel over Ethernet) is the latest punch from Fiber-Channel camp to sustain the longevity of Fiber Channel technology and push back IP SAN.

FCoE will give the servers the Ethernet utility while the servers still get the FC utility for disks. Once FCoE standard is out, we'll be able to see who's clearly the leader in the merge of two camps (IP/Fiber Channel).

So far, Ethernet defeated all other networking technologies and became the Internet. Will FCoE be able to replace the Internet? No, it won't. But FCoE may find the place in the hearts of large corporations while high-speed, high-throughput with quality of service (QoS) are more important than cost.

The new battle between technologies is just begun. Let's watch. Do not adopt FCoE until there is a clear sign that FCoE is going to win. Ethernet defeated ATM, etc before. It may be able to win again.

Consumer Storage Service Provider - the Marketing

Consumer Storage Marketing

When we think about Marketing, we think about press releases, launch events, newspaper ads, and even television ads. In the world of the Internet, especially the ecosystem that CSSP is in, marketing is different.

At present, Google is the main player in Internet Marketing enabling people with the ability to find and locate such products/services they never heard of. Since Google is streamlining most of the Internet Marketing. What a start-up CSSP should do? Paying millions to Google may help with Search Engine rank or Page rank, however, the ability to convert trial customer to paying customer is all CSSP's job. Search Engine Optimization (SEO) may be a good way to increase traffic to CSSP's site. We'll discuss about that later on.

First thought about CSSP Marketing

Before start paying Google for ads, CSSP shall think through Marketing strategy for a start. Selling in a market filled with competitors (CSSP) does not help one very much to focus on the product alone. What motivates a customer to pay for CSSP service?

What is Perception?

Wait for a minute, a perception? Most may argue. However, when you look closely at how you buy your new mobile phone, your new car. It's all about perception - a perception that you get what you want. What perception has anything to do with buying CSSP's service? If a customer does not think that CSSP is providing the right service at the right price, he/she will never sign up. What could CSSP create its perception to user that it brings value that is higher than others?

Offer is the key. Offer is what CSSP customer thinks he's getting. For example, why sell an Online storage to a customer like others. While we normally sell 'ability' to store much more data with high bandwidth. On the other hand, this could well be 'selling Bittorent space' for customers who like to download movies from the Internet. Now CSSP does not sell online storage but sell ability to download, upload Bittorent (the offer).

There are people out there who like to upload/download files at a much more quicker pace. They can opt to rent a space with CSSP and let all the downloads coming in from the Internet. Once the download is completed, they could bring their hard drives to CSSP to copy all the data. This is for customers outside of the US where DSL speed is still low.

As for perception/offer pair for other businesses, CSSP could offer 'centralized' e-mail system. Currently most businesses that run their own e-mail server. This means that their e-mail traffic is coming in through their expensive leased line along with their Internet (web) traffic. In some cases, an important e-mail may not arrive in a timely manner because it's stuck in leased line traffic.

The way to ease this problem, CSSP could offer 'centralized' storage for e-mail server. So all e-mail messages will only come to CSSP data center. If a user is waiting for an important e-mail, he/she could check e-mail and see whether it has arrived, then download to his/her Inbox folder. This saves Internet traffic to interfere with important business. CSSP could also provide storage space for Web server in a similar manner.

This offer could be the offer to Ensure important e-mails will arrive in a timely manner. Customers are not going to afraid that its Internet traffic will push its important e-mail to arrive. This increases efficiency and reduces turn-around time to respond to an important client's request.


Next I'm going to write about Bittorrent Provider model. I've come to think of it as a way to exploit unused network bandwidth/storage utilization. After Bittorrent- I'll write about Nirvana of Storage in IT-doesn't-matter-related series.

Consumer Storage Service Provider - Part 1

I've written much about Enterprise Storage Service Provider. The reason behind storage service for corporations are: 1) an urgency in corporate world to backup and be prepared for disaster especially after 9/11, 2) public companies are looking for ways to be resilient even after a plane crashing into their headquarters (how ironic).

Consumer Storage Service provider is for mass market. These providers are typical web start-up. They do not require a large investment or VC backing like Enterprise Storage Service Provider. But they need a team of dedicated professionals and reliable storage. What do these Consumer SSPs need?

Since Consumer SSP will serve thousands or tens of thousands of subscribers. They can not afford to sign-up one subscriber at a time, nor they can afford to customize their service to one particular subscriber. They need to automate everything from sign-up, service provisioning, billing, and even service termination. The sample web application from my earlier blog is merely service description. The back-end system is a wholely differnt ball game.

Today, I'm going to elaborate the setup for a Consumer Storage Service Provider. What do you need to start a CSSP (other than a business plan)?
1. Storage - to host customers' data
2. Web server - to sign-up, service subscribers - the door to storage for subscribers
3. Backend server - to charge subscribers, to activate subscribers that register and pay for service, to terminate subscribers that do not pay or stop or pause the service.

#1 - Storage - Wasabi Systems may have a single point of failure, as long as your service level agreement is not 4-9s (99.99%), Wasabi Systems product would do. It may be wise to double up the storage for premium subscribers (if budget is not to tight).

If SLA is 99.99% - you will need other iSCSI storage systems. There are many players out there that provide iSCSI storage. Some of these new players are Huawei, Rasilient, and Rackable Systems. These companies equip their offerings with dual controllers (no single point of failure). So they provide good enough systems for Consumer SSP with 99.99% SLA.

#2 - Web server - any Intel-based or AMD-based Linux or Windows server will do. My sample Storage Service web application runs on Apache/MySQL. If there are many subscribers, Consumer SSP will need to setup a dedicated MySQL server with multiple Apache-based web servers. In the beginning, you only need two servers.

#3 - Backend server - same as above, PC-based or server-based running any operating system will do fine. Sign-up, activation, billing, and termination can be done automatically on this PC.

Sign-up is a feature on the web application. However, activation/termination are backend software's features. After sign-up request is being submitted. A transaction is delivered to the backend server. Then backend server requests authorization for subscribers' credit card before proceed to activate the service. Once payment is confirmed, backend software will activate that particular subscriber. But if payment can not be processed, charge collector (backend software) will send a notification to that subscriber and then send a block to non-pay accounts.

As for billing, administrator has to verify data integrity and create revenue report. Typically Consumer SSP will sign a revenue-sharing deal with ISPs in that market. So revenue report is required for bill settlement with ISPs.

Backend systems (workstation) for Consumer SSP will need the following:
3.1 Intel-based or AMD-based PC workstation
3.2 Windows or Linux operating system
3.3 Mysql installation on Backend systems
3.4 Replication setup from a production server/storage -> fail-over/reports engine server
3.5 Once everything crashed, this client can be used to do fail-over/reports engine server

I'll talk about Consumer SSP marketing in the next blog.

Web Application to deliver Storage Service

There are a number of Storage Service web applications out there that are currently serving customers. These applications are normally built by frameworks ranging from .NET and Java. There are some new ones that are emerging from new technology such as Ruby on Rails. The methodology, the framework, or the language that were used to built the application does not matter. The features for Storage Service web application are the most important.

Our Storage Service web application does multiple things. The following use-cases were thought at the beginning:
1. Self sign-up process
2. Once registration is confirmed, a new user can log-on and start uploading files to backup.
3. There are two modes of uploading files -
3.1 Single file upload
3.2 Multiple file upload
4. Once the files are already uploaded, user can start to manage directories and file location. These files are able to be downloaded if the local copy of these files are lost.
5. User can opt to share some of his/her own files for other users within the same systems such as news, tips, or bulletin board - this is called Public Sharing
6. User can also share the files with a close group of people based on the number of downloads or the length of time - this is called Private Sharing
7. If a user wants to protect his/her files, he/she can encrypt any files with AES 128 bit.


We'll start examining from the top why and how are things being done:
1. Self sign-up process
- since we don't take orders manually over the phone, we need users to register for the service and find a way to pay for it themselves. Once payment is confirmed, they will receive a notification e-mail to start as the first day of service. The billing cycle is based on the start date/time of the service.

- current sign-up process can be done either from the web interface or SMS to our SMS engine.

- we could turn it off originally to provide free service promotion for certain period of time. We plan to launch a marketing campaign in January 2007 together with True for its SME customers at True SME World Exhibition. This could be easily setup.

2. Confirm registration with a notification e-mail
- once payment is confirmed, we automate the notification via e-mail. This could be engineered into an SMS notification to user instead. The notification mechanism will be through the same channel. If user registers for the service via web interface, notification will be via e-mail. However, if user registers for the service via SMS (Short Message Service), notification will be via SMS.

- setting up payment method via SMS may seem harder. With Northstar Mobi service, in the near future, user can opt to choose Mobi payment method as well.

3. Upload files
3.1 Single file upload - user browses for a file that needs to be uploaded to keep. Then user clicks confirm to send it over. The dialog box will show the percentage of that upload until it's compelted. To prevent uesr from confusion whether the file has been successfully uploaded. User can also check from the main page after login whether the files that were uploaded are there or not.

3.2 Multiple file upload - user looks for files that need to be uploaded. Multiple file upload dialog box will pop up and wait for the user to select files, then upload. However, uploading an entire directory is not supported in this level. User needs to enter that directory to select all the files in that directory.

4. File management
- user can delete, move, and copy the files on the server through a web page. User can also create directories, sub-directories and so on. It is fairly easy to navigate even though user is not IT oriented. This makes it easy for user to create good document repository.


5. Public Sharing
- user can choose files that are needed by others within the same system. Public sharing will copy those files to a public directory. Every user on the same system could browse this public directory for files, news, tips that they want.

6. Private Sharing
By number of downloads - user can select files that can be downloaded a certain amount of time. This is in the case the user wants 4 of his/her friends to download a picture. This user could set the number of allowable download to 4. Our Web application will store the number of downloads permitted by the owner of the files and the number of downloads left to ensure the number of downloads rule still applies. A failed download session is counted as one download session to prevent any data confidentiality.

By length of time - user can select the length of time that these selected files can be downloaded. User's friends can download as many times as they want. But once the time limit is up, these files are not download-able.

7. Encryption

Security is put last on this list. It's the most important issue for most customers. We use 128-bit encryption either AES, DES. More encryptions are on the way such as Public key cryptography. User can upload his/her sensitive information for backup. However he/she may not have the peace of mind that his/her files can be viewed by someone else.

Before starting using this feature, user needs to create a key to encrypt these files. Key generation is done automatically once a user registers for the service. Then user needs to store the key somewhere safe. If the key is lost, it is almost impossible to bring the files back. The latest computers will spend millions of years trying to break these code. We have tested with all Microsoft Office files (MS Word, MS Excel, MS Powerpoint, Visio) and they all work beautifully when we extract the files back to its original stage.

All these features were built more than 2 years ago. We are excited to see its growth and ROI. If you need our help, send us a note. The sample contracts are coming to this blog soon. If you need any help setting up a Storage Service provider, Storage Service web application, and even making test subjects, let me know. I can help you to be up and running in a few weeks.

Next entry will talk about legal contract and setting up an operation. Have a lovely weekend, everyone.

Storage Service Provider - Current State of the Business and Mass Market Storage Service

The current state of business for Storage Service Provider is not so bright for some. Especially SSPs in the United States with Fiber-Channel systems are starting to close down. Such as this. The downsides of Fiber-Channel storage system are point-to-point, inflexible, and too costly.

Point-to-point architecture is problematic for replication. Any replication setup could only be from one source storage to one or more target storage(s). Replication requires Fiber connectivity from source to target with DWDM equipment on both sides. iSCSI uses IP, so it could be point-to-multipoint with ease.

Fiber-Channel system is inflexible in such a way that once you need to move the target storage, you need to plan to move the fiber connectivity first. Then you need to move your DWDM equipment along with the target storage system. iSCSI is IP-based. You don't need to worry about re-cabling your fiber connectivity. Just shutdown the target storage, move to the new site with the new IP address. Turn it on, you're back in business.

Costly: renting a dedicated dark fiber between source storage to target storage is very costly.

These are the reasons why Fiber-channel-based Storage Service Provider can not sign up enough clients to be profitable. They need high-profile clients who are willing to pay a lot of money. These clients are typically worried about their confidential information. It is almost impossible for banks to sign up this service. So SSPs in this space never get enough clients to be profitable.

The only way for non-profitable storage service providers to stay in the business. They need to either increase the average subscription fee or reduce their cost. Increasing revenue by increasing fee might be hard. So reducing cost is the only way.

iSCSI storage systems are typically 30-50% cheaper than its fiber-channel counterparts. In addition, iSCSI architecture utilizes IP network for replication - so it does not have Fiber-Channel inflexibility, point-to-point architecture, high-network cost. There are more low-cost iSCSI players popping out on a daily basis.

Current iSCSI systems are very reliable such that some of it could be used in lights-out operations. Our system at True runs with no problems for over a year and a half. There is a sample on the newer iSCSI companies that offers low-cost solution for new SSPs.

The first company that I'd like to introduce is Wasabi Systems. I heard about its name from open iSCSI group. People on open iSCSI list tried to use Wasabi software to build their own iSCSI arrays. The beauty of Wasabi is that they only provide DOM (ROM+software) module that you can plug into IDE port on your PC/server making your PC/server an iSCSI array.

The cost per DOM is very economical - in hundreds of dollars. So a 3TB Wasabi iSCSI array with NAS capability could be built with two thousand US dollars. The only downside of Wasabi is that it does not have cluster capability at the moment. So Wasabi array will have a single point of failure at the controller (your mainboard/CPU/DOM). For most non-critical use, Wasabi storage is perfect. Any geeks who know how to build a PC from scratch will find Wasabi easy to build.

For services like Verizon's that provides massive space for users at low cost (5GB for 5 USD per month) could be built with Wasabi software. Let's do a quick calculation for return on investment.

First, we need to calculate the usable space from 3TB storage. Let's use RAID-5 on 3TB, so we'll have 2.5TB usable. But 2.5TB from 5 hard disk drives does not equal 2.5TB in storage world. It's 2,500,000 MB = 2,500,000/(1024*1024) = 2.38TB.

Assumption:
1. Invest 2,000 USD for 2.38 TB usable.
2. Monthly cost for hosting is 100 USD (3TB storage and a webserver)
3. Monthly cost for one webserver is 100 USD (lease)
4. Salary for a part-time administrator is 1000 USD (shared with IDC)
5. Price point is the same as Verizon's service - 5GB for 5USD/month

Scenarios:
Scenario 1: If I can rent all 2.38TB in the first month, I'll get 2,380 USD in the first month.
After paying for monthly cost (1+2+3+4), I still have 1,180 USD left towards 2,000 USD investment. My return on investment is 2,000/1,180 = 1.6 months

Scenario 2: If I don't pay for #4, I slave myself without pay. My monthly cost is 200 USD. While I start ramping up the service, I sign up 40 users every month. Then the first month revenue is 200 (40*5), second month is 400, ... 12th month is 2,380. I'm going to be 200 USD profitable in the second month (revenue (400) - cost (200)). My return on investment is 5 months.

SSP should engineer their ROI to be somewhere between 1.6 months to 5 months. If all goes well, after the 5th month, SSP can start collecting pure profit to build additional arrays to expand the service. The beauty of this is that owners don't have to be there (lights-out operation) and the money keeps flowing in.

In the end, creating a mass market service from a low-cost storage system is very profitable. If SSPs could not sign up 40 users a month, they need to re-evaluate their marketing plan again. It's either they target the wrong audience. Or the market they want doesn't exist.

If you need help to build Wasabi system or web application to provide Mass-market storage service, let me know. We can build one for you. We also will share sample contracts with end users to the people we help.

I'll elaborate how the web application works in the next blog.

Good luck, all newbie SSPs.

Storage Service Provider - Enterprise Storage Service Provider (2)

As I mentioned, I'm doing a lot of things currently so what i promised to write two weeks ago is just about to be out today.

iSCSI is going to be the main driving force behind a lot of ESSPs since it's based on IP. Yes, Internet Protocol. Plain and simple. IP won the battle with all competing technologies namely token ring, ATM, etc. TCP/IP has a lot of nice features - block re-ordering (in case one packet arrives before the other, TCP/IP stack will put them in order), link aggregation (ability to put 2 links to act like 1 - or having 2x1Gb port act as 1x2Gb port), and more.

In addition, TCP/IP is not point-to-point like Fiber Channel, it could be multi-point to multi-point. So you can have multiple data centers and then you decide which combination of disaster recovery scenario you'd like to do.

Let's see if you have 3 data centers with iSCSI in each location across the US (Palo Alto (CA), Chicago (IL), New York(NY)). You can have all three iSCSI systems replicate data to each other to ensure that you have 3 copies of data.

So you'll have the following replications: CA->IL, CA->NY, IL->CA, IL->NY, NY->CA, NY->IL. What if one day your risk pattern changes? You can decide to add another data center in Timbuktu (TI) then you stop all 6 replications. Then do 3: CA->TI, IL->TI, NY->TI. With Fiber Channel technology, you need months of planning. With iSCSI, you only need a few days. If you already put iSCSI storage in TI, this exercise could finish in a day (depending on the network bandwidth).

Let's put another interesting event, what if CA site was under earthquake. Your CA offices can reroute their network traffic to TI while CA data center is under restoration. Then you setup an interim disaster recovery site in Las Vegas (NV). You can restore data from TI->NV. All this depend on the fact that you have enough network bandwidth among sites.

This is why ESSP business is very interesting for companies that own a lot of network bandwidth! They can sell you bandwidth with managed services offering such as managed storage service, managed backup service, managed disaster recovery service, managed business continuity service.

As one of my wise friends said "Outsourcing is the way of the future". Amen to that, my friend.

We'll talk about how can we get these outsourcing opportunities. (not tomorrow but soon). Have a nice day.