The rule of done

My college roommate Dale reminded me of an important rule of thumb that I use every day. I call it the Rule of Done. Here is the rule:

• “It is not a matter of wanting, it is a matter of doing!”

Self-talk is a wonderful thing. We all do it throughout our waking moments. There is a little voice in the back of your head telling you things. Sometimes I say to my kids “You need to listen to that little voice.” Well I need to clarify something about this piece of advice – you need to listen, but you don’t always need to go along with the input that your little voice is giving.

Here’s an example – the alarm goes off and you are lying there in bed.  The little voice says “I don’t want to get up.” This is an example of one of those places where you need to apply the Rule of Done – “It is not a matter of wanting, it is a matter of doing.” You may not want to, but you get up anyway.

Here’s another example – you know you have an assignment due in a few days. You realize that you need to start working on it so that you can deliver a high-quality result. That little voice might say “But I don’t want to work on it now, I will put it off until tomorrow.” – There is that tip-off phrase “I don’t want…” again. So apply the Rule of Done. It is not a matter of wanting, it is a matter of doing. You overrule that little voice in your head, and you “get-r-done.” You stop arguing with yourself and “just do it.”

Try this out in your every day life and I think you’ll like the results. Tomorrow when that little voice in your head says “I don’t want to work out” say right back to it “It is not a matter of wanting, it is a matter of doing.” Or when that voice says “I don’t want to cold-call that prospect” or “I don’t want to have that conversation” or “I don’t want to….” Apply the Rule of Done. You will find that you can save yourself a lot of time and a lot of angst by following this rule.

Dale and I had a great time rooming together because we and our roommates shared a respect for the Rule of Done.

Oracle RMAN is the SLA “Tax man”

In prior blogs, I’ve proposed that many organizations are paying “taxes” that they shouldn’t have to pay. These taxes are in the form of wasted costs from owning and running extra server headroom just so that performance doesn’t become unacceptable during data protection operations.

Databases are a great place to show this because the underpin most business-critical systems out there. I showed some specific examples with SQL Server, but just to prove that this is not just a Microsoft problem, let’s take a look at Oracle.

The Backup Tax – Oracle style:

Oracle RMAN is the default utility for performing database backup, restore and recovery of Oracle databases. Here are some results quantifying the impact of RMAN backups in an Oracle database environment, this is for a 2-Node Oracle 11g R2 RAC cluster running a TPC-C style transactional workload.

There are a couple of key points to note.

• Even though this was a lightly loaded system, CPU Utilization increased 30-40% for the entire duration of the backup.
• Response time and Transactional performance see a negative spike for the first couple of minutes of the backup window.

Conclusion: You have to pay the “tax man” for Oracle when you use RMAN on your Oracle host.

For more info on this testing see: http://dell.to/xAsNCa

Now it is time to hear from you. What is your experience with SLA taxes?

Less Aileron… Sooner!

It is interesting how things you learn in one area of life can help in others. In one of my early blog posts, I related some wisdom from my first flying instructor Harry. Here is another gem from Harry that I’ve found most helpful in other parts of my life. It may take some time to sink in for you… but think about it.

One of the hallmarks of a good pilot is how they can smooth out a ride, and precisely control their aircraft in maneuvers. I was flying with Harry one day and clearly was working hard to keep the plane on track while following his commands during the lesson. Harry didn’t talk much, his philosophy was to let the plane do the teaching. On occasion he would impart a key piece of advice so I always listened carefully to what he would say. On this particular day, after watching me work too hard at the controls he said: ”Hey Bob, try this: A little less aileron…Sooner!”

The ailerons on a plane are control surfaces on the wing. Here is a picture:

They are moved by the pilot, along with the rudder and the elevator to control the flight of a plane.

Essentially his input to me was this: I was using large, abrupt movements of the control surfaces. He wanted me to start more quickly but use subtler and more gradual control movements. It took me a while to apply what he said, but after hours of practice, I learned to slightly anticipate the need for control inputs and make the adjustments more progressive – quickly adding a little and then a little more or taking away a little at a time.

How does this apply to other areas of life? I’ll try a couple, and I bet you can come up with a lot more.

• I coach ski racing on the weekends. As skiers work to control their turn shape, one of the variables is edge angle – the amount the ski is tipped relative to the snow surface. This concept works great here – “a little less edge – sooner!” Perfect for smoothing out jerky turns or improving hold on ice.
• As a Dad, I’m trying to help my children develop good study habits. They are realizing that if they study a little each night instead of cramming right before an exam, they do better in school. Imagine telling your kids ”a little less studying – sooner!”

 So now I’m curious to hear from you, what could you “do less of, sooner” to positive effect?

Reduce your taxes part 4 – a SAN solution

In the first blog of this series I discussed how some organizations are over-provisioning their server resources in order to balance the competing demands of performance and data protection SLAs. Parts 2 and 3 showed examples of actual SQL Server database configurations being tested to measure the impact of backups and snapshots while maintaining application performance.

Those tests showed two examples of an approximate 40% tax rate for data protection, meaning that 40% of the server processing headroom was sitting idle so that performance SLAs could be met during data protection operations.

How to reduce your taxes – So how can you reduce those taxes? This is where storage comes in to the picture. The reason that meeting service levels requires over-provisioning of your database servers in these scenarios is that server resources are being used for tasks that are best managed by a SAN. SAN-based snapshots have several advantages:

• They are quick to create – taking under 10 seconds instead of the 10 minutes it takes to create native SQL server snapshots for the same workload use-case.
• They have a very minor performance impact that is of no consequence to server provisioning.
• Snapshots can be used as the source for an off-host backup, eliminating backup tax.

Here is an example of actual testing of the performance impact of SAN-based snapshots for the same SQL Server transactional database load tested in parts 2 and 3 of this series.

As you can see, CPU utilization and transactional performance are minimally impacted by the SAN-based snapshot operations. The “Smart Copy” terminology refers to the fact that these are application-consistent snapshots which have been created by momentarily quiesceing the database and flushing buffers so that a completely valid recovery point for the database is created. This is done with Microsoft VSS, and took between 4 and 8 seconds for the configuration tested.

Conclusion: By using the data protection capabilities of your SAN, you can ensure more efficient use of your IT resources and reduce your taxes to achieve IT Efficiency.

For more information on the testing that was done please refer to:  http://dell.to/w35fsv

Now it is time to hear from you, what is your preffered “Tax” reduction strategy? Are you using SAN data protection?

Reduce your taxes installment 3 – The SQL Server Snapshot Tax

In the first of this series I propose that some organizations are over-provisioning their server resources in order to balance competiting performance and data protection SLAs. In part 2 we examined the “backup tax”. Now let’s look in detail at another example of this, “the snapshot tax” for establishing on-disk recovery points by doing native SQL Server snapshots.

Challenge: The Snapshot Tax – The following graph shows the results of testing snapshot processing on a transactional (TPC-C style) workload running in SQL server. It illustrates a 40% tax associated with establishing on-disk recovery points with the native SQL Server snapshot capability.

There are several observations from this data that should cause concern to those of you who would like to reduce your IT taxes:

• Creating native recovery points impacts performance – average response times and transactions per second see a significant negative impact.
• Creating a recovery point takes a long time – in this scenario about 10 minutes. So if you want to keep one recovery point per hour (RPO), around 1/6^th of that hour will be spent with degraded performance.
• The overall impact becomes cumulatively worse as more recovery points are kept.

Conclusion: Attempting to meet service level agreements for performance and data protection by establishing and keeping 5 hourly recovery points with native SQL server snapshots requires at least 40% over-provisioning of database server resources. This is an exorbitant 40% tax rate that you shouldn’t have to pay.

For more information on the testing that was done please refer to:  http://dell.to/w35fsv

Let’s hear from you. What is your RTO? Is one hour reasonable for recovery points for SQL Server transactional databases? What rate do you pay for snapshot taxes?

Reduce your taxes # 2 – The SQL Server Backup Tax

In the first of this series I discuss that some organizations are over provisioning their server resources in order to balance competing performance and data protection SLAs. Let’s look in detail at one example of this, “the backup tax” for doing native SQL Server backups.

Challenge: The Backup Tax – The following graph shows the results of testing of backup processing on a transactional (TPC-C style) workload running in SQL Server. It illustrates one example of a tax rate of 40%.

The graph of the results shows that approx. 40% of the CPU cycles provisioned for SQL server must be reserved in order to make sure that service levels for performance are met during backup with the SQL Server native backup utility.

Conclusion: Simultaneously meeting service level agreements for performance and data protection while using the native SQL server backup requires at least 40% over-provisioning of your database server resources. And this assumes that you’re willing to run your database server at 100% utilization during the backup window. This is an exorbitant 40% tax rate that you shouldn’t have to pay.

For more information on the testing that was done please refer to:  http://dell.to/w35fsv

Let’s hear from you, what do you pay for backup taxes? What is your tax reduction strategy?

Reduce your taxe rate to achieve IT efficiency

Yes it is that time of year again; we all start thinking about taxes. I know what you’re thinking; “What does this have to do with IT and enterprise efficiency?”

It comes down to managing SLAs. As you design your IT environment you need to be cognizant of Service Level Agreements. What are the expectations of your users for performance? What is the agreement you have with them about response time? Also what is the agreement you have with management about data protection, what are your Recovery Point Objectives and Recovery Time Objectives?

Databases are a great place to explore this balancing act.  Databases underlie most of our business systems. As you size your database servers, you must ensure that SLAs for performance are met while at the same time making sure those SLAs for data protection are met. It turns out that this usually results in database servers being over-sized to deliver acceptable performance AND accommodate for the processing load of establishing recovery points. This is a “tax” that almost every IT organization is paying, and with the right storage strategy you can reduce, and even eliminate most of this tax.

In this series we’ll look at some lab testing of transactional database systems (TPC-C style workload) under different scenarios to determine what that tax rate might be.  I think you will be surprised.  The data shows on the order of a 40% tax rate being paid by IT departments. I mean by that on average servers must be sized so that 40% of their headroom lies dormant in order to make sure that performance and data protection SLAs can be met. And this assumes that IT departments are willing to use all available headroom and run their servers at 100% utilization during data protection operations, which is rarely the case in the real world.

It is time to hear from you. What is your experience with this issue? Are you paying these taxes? Would you like to reduce your tax rate? Can you share any specific examples?