As I sat there, I ended up reflecting back on 2025. I found myself gravitating to these three questions:

• What went well this year?
• What did I learn?
• What should I focus on next year?

If you’re a leader, I’m guessing you rarely get this kind of thinking time during your workday. I know I don’t. Our calendars are packed with calls, team meetings, and those “quick questions” that turn into two-hour troubleshooting sessions.

But here’s what I’ve learned: the quality of your strategic decisions is directly tied to the quality of your thinking time.

And thinking time doesn’t happen by accident. You have to protect it.

What Went Well This Year?

When I asked myself this question, I didn’t think about our biggest projects or flashiest achievements. I didn’t think about when we migrated almost 2,000 databases as part of an upgrade project. Or the performance tuning we did that resulted in a $36,000 reduction in annual Azure spend for a client.

Instead, I thought about the relationships we strengthened. The trust we built with clients. The problems we solved before they became crises.

For you, this might look like:

• The audit that went smoothly because your security documentation was solid
• The successful disaster recovery test that was possible because you kept refining the process
• The team member you mentored who’s now ready for more responsibility
• The support resources you provided your team through a trusted partner

These aren’t always the things that make it into board reports. But they’re the foundation that everything else is built on.

What Did I Learn?

This year reminded me of something Eisenhower once said: “Plans are worthless, but planning is everything.”

The need for planning cannot be overstated. It’s critical. Even if the plan doesn’t always work out the way you intended.

The plan itself wasn’t the point. The thinking I did while creating the plan was the point.

Because I’d thought through our capacity, our ideal client profile, and our service delivery model, I could adjust quickly when reality didn’t match my spreadsheet. I knew which opportunities were a good fit for us and which ones to let go. Because we’ve intentionally built a small but incredibly talented team that genuinely wants to see our clients succeed, we were able to identify and create ways to help them.

I watched the same dynamic play out with clients. The institutions that had documented their SQL Server environments, tested their disaster recovery plans, and mapped their compliance requirements adapted quickly when needed. They were positioned for success even when the unexpected happened.

Planning isn’t about predicting the future. It’s about building the muscle memory to respond when the future surprises you.

What did you learn this year about planning and adapting? Maybe it was:

• That your three-year technology roadmap needs quarterly reviews, not just annual ones
• That the disaster recovery plan sitting in a SharePoint folder isn’t the same as a tested DR plan
• That “we’ll address that next quarter” eventually becomes “why didn’t we address this sooner?”
• That having an expert on call beats having a plan to find an expert when something breaks

These lessons matter. Write them down. They’re not just hindsight—they’re your blueprint for better decisions ahead.

What Should I Focus On Next Year?

For me, the answer was clear: I need to help more financial institutions and healthcare organizations understand that they have options. Most CIOs think they have two choices for database management: hire a full-time DBA (expensive and hard to find) or make do with whoever can “figure it out” (risky and unsustainable).

There’s a third option: fractional DBA services that give you expert oversight without the full-time price tag.

For you, your focus might be different. Maybe it’s:

• Finally getting your SQL Server environment documented and audit-ready
• Building a disaster recovery plan that you’ve actually tested
• Move a little further along the SQL Server Maturity Curve
• Finding a partner who understands banking compliance, not just databases

Whatever it is, the key is to actually choose something. Not everything. Something. And move toward it. Make progress.

The Power of Quiet Reflection

Here’s the thing about those early Saturday morning moments: they’re rare. And precious.

During the week, we’re in execution mode. We’re responding, reacting, solving, and fixing. That’s necessary work. But it’s not strategic work.

Strategic work requires space. It requires stepping back from the urgent to focus on the important.

So, here’s my challenge to you as we wind down 2025 and usher in the new year:

Block Off Time Just to Think, Then Protect It

Maybe it’s Saturday mornings before your family wakes up. Maybe it’s a long walk at lunch. Maybe it’s 90 minutes with your calendar blocked and your office door closed.

Whatever it is, protect it. The decisions you make during that quiet time about where to focus, what risks to address, and which partnerships to invest in will help shape your entire year.

Your Turn

As you think about the year ahead, I’d encourage you to ask yourself those three questions:

1. What went well this year? Celebrate it. Learn from it.
2. What did I learn? Write it down. It’s wisdom you paid for.
3. What should I focus on next year? Pick one or two things. Not everything.

And if one of those focus areas is “finally get our SQL Server environment to a place where I’m confident, not just hopeful,” let’s talk. That’s exactly what we help institutions do.

If you’re a CIO wondering whether your SQL Server environment is as healthy and secure as it should be, I’d be happy to have a conversation. No sales pitch. Just two people talking candidly about database management. Schedule a time here.

The post Why Quiet Reflection Leads to Better IT Strategy Decisions appeared first on The SERO Group.

Multi-Attach EBS volumes in AWS enable organizations to use Windows Server Failover Clusters with shared storage. Attaching a single EBS (Elastic Block Store) to multiple EC2 instances simultaneously makes them ideal for SQL Server FCIs.

What are Multi-Attach EBS Volumes?

These EBS volumes are single Elastic Block Store volumes that can be attached to multiple EC2 instances. The feature is available for io1 and io2 Block Express volumes. However, Windows instances only support io2 Block Express volumes. Multi-Attach facilitates high availability and seamless failover for SQL Server FCIs (similar to how they operate on-premises).

Prerequisites

Before setting up these volumes for SQL Server FCIs, ensure the following prerequisites are met:

1. EC2 Instances in the same Availability Zone

All EC2 instances that will attach to the EBS volume must reside within the same AZ. This ensures low latency and high throughput between the instances and the EBS volume. This is critical for maintaining performance and reliability in a failover scenario.

2. Supported EBS Volume Types

While both io1 and io2 Block Express volumes support the Multi-Attach option, Windows instances only support io2 Block Express volumes. This limitation is due to the enhanced performance and features that io2 Block Express provides.

3. Regional Availability of io2 Block Express Volumes

Verify that io2 Block Express Volumes are available in your AWS region. Not all regions may support this volume type, so review the Amazon EBS Provisioned IOPS SSD volumes user guide to confirm availability.

4. Additional Networking and VPC Configurations

While this post focuses on EBS Multi-Attach, ensure that your networking and VPC configurations meet the requirements for setting up a Windows Server Failover Cluster. See this link to review the full list of prerequisites for WSFC.

io2 Block Express Volumes

These volumes offer the highest-performance block storage. Key features include:

• Capacity: Ranges from 4 GB to 64 TB, accommodating various workloads.
• Performance:
  • IOPS: Up to 256,000 IOPS per volume.
  • Throughput: Maximum throughput of 4 GB/s per volume

These high-performance metrics make io2 Block Express volumes ideal for demanding SQL Server workloads, especially those requiring rapid data access and high transaction rates.

Compatibility

io2 Block Express volumes are available on all Nitro-based EC2 instances. The AWS Nitro system is a collection of hardware and software components designed to provide enhanced security, performance, and flexibility for EC2 instances.

Enabling Multi-Attach for EBS Volumes

By default, this option is disabled but simple to enable. See this link for instructions on using the AWS Console or Command-Line.

Want to work with The SERO Group?

Want to learn more about how The SERO Group helps organizations take the guesswork out of managing their SQL Servers? Schedule a no-obligation discovery call with us to get started.

The post How to Build SQL Server FCIs in AWS with Multi-Attach EBS Volumes appeared first on The SERO Group.

HADR for binkies? What? And how does this relate to SQL Server?

Sometimes the difference between the HA (High Availability) and DR (Disaster Recovery) can be difficult to grasp. There a nuances and the approaches can overlap. So, let’s consider an analogy – binkies and HADR in SQL Server.

As an infant, one of our daughters was very partial to her binky. She was seldom without one. Day or night, she “needed” her binky.

The Disaster Recovery analogy

We knew that life was better when she had a binky. Every time we left home, we strapped her into her infant car carrier, also known as a her “cry bucket”, and ensured she had a binky in her mouth.

If it was lost during our trip, life would be unpleasant. So we were prepared. We carried extra. At least one, sometimes more than one, just in case the unthinkable happened.

If she somehow managed to lose her binky, if it fell out along the way unnoticed, we would open the diaper bag and retrieve the spare.

That was binky DR. We would have to manually step in to solve the problem. She was disrupted until the fix was in place. But it was better than long periods of screaming about it. We could handle a little disruption and still avoid an all out disaster.

How this applies to SQL Server

The same is true for SQL Server. DR is a way to recover from some kind of disaster. It could be at the data center level, or at a SQL Server instance level. In either case, we need to recover from some kind of failure as quickly as possible.

In SQL Server, that could mean:

• Restoring from a backup. Assuming your backup process is robust, meets your RTOs and RPOs, and is regularly tested, you can apply the various backup files to get you back to whole again. Typically, you’ll restore a full backup, a differential backup, and then a serious of transaction log backups.
• Restore a virtual machine. If the failure affects the instance and not just a database, you may have to restore a virtual machine image. You may be done after that. Or you may have to restore a the backup files as described before to meet your RPO requirement.
• Switch to a stand-by SQL Server. Log-shipping can be a cost effective way to implement a level of DR. If you’re leveraging it as part of DR strategy, to recover you’ll need to make the standby SQL Server the new production SQL Server. You’ll stop log-shipping, recover database on the standby server, and point everything to the new production instance.
• Site-wide DR. Of course, some companies have a site-wide DR plan. If the entire data center is out of commission, they can failover to the DR site in another city or region. Properly configured, that can work very well. Typically, however, you don’t want to do that for a single instance of SQL Server. It’s reserved for disasters on a larger scale.

Disaster Recovery Summary

In all of these cases, manual intervention is required. Someone had to do something to get things back to normal. In our analogy, someone had to dig through the diaper bag to find the spare and give it to our daughter. Until that point, everyone was unhappy.

It’s also worth noting that we checked and double checked that we had a spare binky before leaving home. For SQL Server, you want to regularly test your backups to make sure you can indeed recover from them when needed.

High Availability

Our approach to binky DR when we left home tended to work pretty well. There were times when we weren’t able to recover quickly enough and drew awkward glares from those around us. However, for the most part it was a success.

At night it was a different story.

We would go through the bedtime process – reading a story, rocking for a little while, maybe even humming a lullaby, and gently placing her in her crib. Then we’d tip toe out of the room.

Unfortunately, at some point during the night, her binky would slip from her mouth. And before long, she’d notice. And when she noticed, we all were quickly made aware. One of us would have to get up, search through the blanket and put the binky back into her mouth.

Fortunately, we discovered a better way.

We began putting her in her crib with a binky in her mouth, and a spare one in each hand. That way, when the binky slipped from her mouth, she could quickly and easily replace it with one from her hand. This allowed her to sleep for longer periods and time. And us too!

Binky high availability at its finest.

How this applies to SQL Server

Many mission critical systems depend on SQL Server. For example, Practice Management Systems or Electronic Medical Records in healthcare, Project Management Systems or Job Costing Systems in construction, and core systems in the banking. They all rely on SQL Server. If SQL Server is down, the business is down. And that can be costly.

For those situations, we can leverage High Availability in SQL Server. Two technologies are commonly used.

• Failover Cluster Instance (FCI). With an FCI, one SQL Server instance is effectively spread across multiple Windows Server Failover Cluster nodes. SQL Server runs on the primary node. In the event of a planned or unplanned downtime, a secondary node will start its services, take ownership of the shared resources, and begin servicing application requests. This can happen automatically, often with minimal disruption to users.
• Availability Groups (AG). With an AG, SQL Server is installed on two or more nodes of a Windows Server Failover Cluster. (Strictly speaking the nodes don’t have to be WSFC. Linux clusters can be used.) As with the FCI, if a disruption occurs on the primary node, a secondary node will be promoted to primary and all application requests will be automatically directed to the new primary node. There are a lot of nuances to configuring an AG including synchronous / asynchronous transactions, automatic or manual failover, etc. The point here is that it can be configured to happen automatically and will little noticeable disruption to the end users.

High Availability Summary

One really nice distinction with High Availability is that it can happen automatically, without someone having to do something to make it happen. For businesses running critical applications on SQL Server, this can prevent costly unplanned and planned downtime.

For parents with kids dependent on binkies, this could mean hours of additional uninterrupted sleep each night.

In either case, HA can be really worth looking into.

Want to learn more?

Hopefully this analogy has helped. Here are a few addition resources that may likewise be helpful.

• What’s the Difference in SQL Server FCIs and AGs? – The SERO Group
• High Availability and Disaster Recovery in SQL Server – The SERO Group
• How to Create SQL Server 2019 Failover Clustered Instances in Azure – The SERO Group
• Azure shared disks – Failover Clustered Instances – The SERO Group
• Using VM Snapshots to Backup SQL Server? – The SERO Group
• Availability groups: a high-availability and disaster-recovery solution – SQL Server Always On | Microsoft Learn

If you’d like to talk though how improve your SQL Server’s reliability, give us a call. We’re happy to help.

The post HADR for Crying Out Loud appeared first on The SERO Group.

The right configuration settings are critical. Unfortunately, it’s easy to overlook many SQL Server settings and best practices. The defaults are not always the best.

A proper maintenance schedule for your databases and servers is important. So is regular monitoring. And a sound HADR plan that meets the needs of the business is critical.

Is your SQL Server healthy? A free webinar

In an upcoming webinar, we’ll discuss some commonly overlooked SQL Server settings and practices we uncover during our SQL Server Assessments. These can have dramatic effects on your database server’s performance and reliability. And we’ll share what you can do to fix them and help to keep your SQL Server healthy.

Register here for the free webinar on June 23, 2021.

In the meantime, check out these posts.

• Identifying SQL Server Disk Latency
• Scary and Dangerous Things in SQL Server
• Securing Your SQL Servers, What Should You Audit?
• When Was the Last Known Good DBCC CHECKDB Integrity Check?
• Identify Disk I/O Performance Issues for Your SQL Server Using DiskSpd
• Is My SQL Server Configured Properly?
• How Many tempdb Data Files Should My SQL Server Have?

Want to work with The SERO Group?

Want to learn more about how SERO Group helps organizations take the guesswork out of managing their SQL Servers? It’s easy and there is no obligation. 

Schedule a call with us to get started.

The post Webinar: Is Your SQL Server Healthy? appeared first on The SERO Group.

There are a lot of how-to articles on the internet for setting up Always On Failover Clustered Instances (FCIs) and Always On Availability Groups (AGs). In fact, Luke Campbell created a couple of nice guides – How to Create SQL Server 2019 Failover Clustered Instances in Azure and Azure shared disks – Failover Clustered Instance – SQL Server 2016.

So, I won’t deep dive into the technologies. Rather I’ll share some of their key, high-level differences. The goal is to help you identify the better choice for your environment. Even with that narrow scope, this is a big topic. I’ll touch on the basics here and elaborate on the more advanced aspects in the future.

Keep in mind that there are lots of ways to combine different technologies to improve your HADR capabilities. For example, you can use a FCI to host one of the AG replicas. You can use Distributed AGs in different datacenters. And many other combinations.

There’s a TLDR; summary table comparing the two at the end of the post.

Comparing SQL Server FCIs and AGs

FCIs and AGs aren’t exactly apples and oranges. They have similarities. Both are native to SQL Server. And both can greatly reduce planned and unplanned downtime. However, there are some key differences in their requirements and the way they are implemented. These differences may influence what’s best in your environment. Let’s have a look.

What are the hardware and software requirements for FCIs and AGs?

Let’s begin by considering some of the aspects that may make one of these technologies a non-starter for you – the hardware and software requirements.

Windows Server Failover Clustering (WSFC) serves as the underlying foundation for both FCIs and AGs. At least in the Windows world. Pacemaker is used for clustering Linux instances. Here, I’ll refer to them collectively as WSFC.

FCI requirements and operation

With an FCI you’re essentially installing one instance of SQL Server across two or more nodes of a WSFC. Only one node is active at a time, the primary node. The secondary nodes are available, but not active. Clients connect to the Virtual Network Name and do not need to know which node is currently serving as the primary.

Should the primary WSFC node become unavailable, the SQL Server services are started on another node and it becomes the primary node. All databases, logins, jobs, etc, that are part of that SQL Server instance are available on the new primary node. That’s good. FCIs provide instance-level protection.

To make this possible, FCIs require shared storage. The primary WSFC node owns the shared resource. When it fails, another node will take ownership of the resource and then start its SQL services. The SQL binaries reside on each WSFC node, not the shared storage.

FCIs are available in both Standard Edition and Enterprise Edition of SQL Server. From a licensing perspective, FCIs are a single instance of SQL Server since only one node is actively serving clients. (Note: a qualified licensing expert should be consulted to ensure you fully understand the SQL Server licensing. It can get quite complex.)

AG requirements and operation

AS with FCIs, SQL Server is installed on each WSFC node that is part of an AG. However, unlike FCIs, the SQL Server service runs continually on each node. One of the nodes is designated as the primary replica and all other nodes are secondary replicas. Clients connect to a Listener that directs the connection to the primary replica.

Each WSFC node has its own copy of the AG databases. No shared storage is required. When changes are made to the primary replica, those changes are propagated to the secondary replicas. The increases the total storage capacity required. A two-node AG doubles your storage requirements. A four-node AG increases your storage requirements by a factor of four.

AG can be configured for HA (more on that later). Secondary replicas in HA mode typically do not require licenses.

When the primary replica in an AG configured for HA goes down, one of the secondary replicas becomes the new primary. The Listener directs all connections to the new primary replica.

AGs provide database-level redundancy. This means that all databases that are part of the AG will failover to the new primary replica at the same time. Instance-level objects are not part of the AG. So, logins, SQL agent jobs, etc, are not automatically available on the secondary replicas.

Basic Availability Groups are available in SQL Server Standard Edition. Basic AGs are limited to a two-node system with a single database. AGs in SQL Server Enterprise Edition can have multiple databases that failover together and up to 8 replicas per AG.

Is it HA, DR, or both?

There can be some overlap in what’s considered High Availability (HA) and what’s considered Disaster Recovery (DR). Though few would argue that the terms are interchangeable, some technologies can span across both terms depending on how they are implemented.

Generally speaking, HA means the databases remain online and fielding requests despite what would otherwise be a disruptive event. So, users don’t experience noticeable downtime due to unexpected events such as a hardware failure, or during normally scheduled maintenance windows.

DR, on the other hand, is typically a response to a more catastrophic event that could affect more than the SQL Servers. Fires, Tornados, flooding, hurricanes, etc, can hit a datacenter or even an entire region. Ransomware can make its way into a network. The question is how quickly can your team respond to such events?

FCIs and HADR

FCIs are generally considered an HA technology. They are designed to keep the databases online during unexpected events. FCIs can span multiple subnets, datacenters, etc, to help with DR, but they are primarily used for HA purposes.

AGs and HADR

AGs have couple of modes that are frequently used for different purposes. If the nodes are configured for Synchronous-commit mode, each transaction is held open until the secondary replicas are updated. This keeps the replicas up to date at the possible expense of performance. This is good for HA scenarios.

With asynchronous-commit modes, the transactions complete as soon as the log file is written on the primary replica. This allows for some lag between the primary and secondary replicas. Should the primary unexpectedly go offline, some data loss can occur. Asynchronous-commit mode is often used in DR scenarios, when secondary nodes are in a different datacenter or region.

What about RTO and RPO?

No HADR discussion is complete without Recovery Time Objectives (RTOs) and Recovery Point Objectives (RPOs). How do FCIs and AGs stack up on RTOs and RPOs? (See Where to Start with Disaster Recovery in SQL Server for a description of RTOs and RPOs)

FCIs and RPOs / RTOs

Recall that each WSFC node has access to a shared storage resource. Only one node at a time owns the resource, but all nodes have access to it should they become the primary node. The system and user databases are located on the shared storage resource. As a result every transaction committed on the primary node persists after a failover to a new primary. There should be no loss of data during failovers.

During a planned failover, dirty buffers are written to disk on the primary before the role is transitioned to a new primary. During an unplanned failover, committed transactions that never made it to the data file are rolled forward during the start-up recovery process. Failovers are frequently very fast, however for busy systems, it can time some time.

AGs and RPOs / RTOs

As mentioned before, AGs can be configured in synchronous-commit or asynchronous-commit modes. Synchronous-commit mode can further be divided into “planned manual failover only” or “automatic failover”.

AGs configured with Synchronous-commit with automatic failover will, as the name implies, automatically failover to a secondary replica should the primary become unavailable. This process generally doesn’t take too long. Of course, you’d want to test in your environment to know what to expect.

For secondary replicas configured for Synchronous-commit with planned manual failover only, failover requires manual intervention. Once initiated, the failover typically occurs relatively quickly.

Asynchronous-commit replicas also require manual intervention and could experience some data loss since the secondary replica may be slightly out of sync with the primary.

How difficult is it to manage FCIs and AGs?

FCIs provide instance-level protection. Once installed and configured, FCIs are essentially a single instance of SQL Server spread across a WSFC. As a result, you don’t have to worry about ensuring logins, jobs, etc, are consistent among the WSFC nodes. If you’d like to add or remove nodes, use the SQL Server Setup program.

AGs, on the other hand, require a bit more effort. Since AGs provide database-level protection, you’ll need make sure that important instance-level objects that AG databases depend on are available on the secondary replicas. For example, primary replica logins are not automatically created on the secondary replicas. You’ll need to do the same for maintenance jobs. We typically add a step to the beginning of each job to determine whether the job is on the primary replica or a secondary replica. If the job is running on the primary, continue. Otherwise, don’t do anything.

What about patching FCIs and AGs?

Both FCIs and AGs can minimize downtime due to patching. Although the exact details vary, you can apply Windows and/or SQL patches to secondary nodes/replicas, manually failover the services so that the newly patched system becomes the primary node/replica, and then apply the patches to the remaining nodes/replicas.

This approach limits downtime to the time it takes to failover.

Are secondary nodes/replicas available for production use?

It can be interesting to think about using the secondary nodes/replicas to offload work from the primary node/replica. With FCIs this is unfortunately not possible. In fact the SQL Server service isn’t even running on the secondary nodes. So, the secondary nodes aren’t available for other purposes.

With AGs, you can configure secondary replicas for read-only access. This allows actions such as backups to be performed on the secondary replica. Additionally, you can offload some resources-intensive reporting queries to the secondary replica. Note, additional licenses are required if the secondary replicas are used for production proposes.

How do FCIs and AGs compare?

So, what’s the Difference in SQL Server FCIs and AGs? The following chart summarizes the differences.

* See the SQL Server 2019 Licensing Guide available on the Microsoft SQL Server licensing page.

Which is better, FCIs or AGs?

Is one of these two technologies better than the other? Yes, almost certainly. But, which one? That’s really going to depend on what your objectives are, on what technologies and licenses you have in place or can afford, and on how your SQL Servers fit into the broader HADR conversation.

The important thing is to take the first step. Start having the conversations.

In High Availability and Disaster Recovery in SQL Server, I also share a couple of other basic DR options – Backups and Log Shipping.

Want to work with The SERO Group?

Want to learn more about how SERO Group helps organizations take the guesswork out of managing their SQL Servers? It’s easy and there is no obligation. 

Schedule a call with us to get started.

The post What’s the Difference in SQL Server FCIs and AGs? appeared first on The SERO Group.

There are a couple of easy ways to check for disk latency issues in SQL Server: using the sys.dm_io_virtual_file_stats DMV and using the dbatools.io Test-DbaDiskSpeed command. Let’s look at each.

Check disk latency using DMVs

Way back in SQL Server 2005, Microsoft introduced the sys.dm_io_virtual_file_stats Dynamic Management View (DMV). This DMV reports disk read and write activities for data and log files.

For many years, I’ve used a query based on one from SQLSkills’ Paul Randal. I’ve simply added a column to help quickly categorize/interpret the latency values. This query can be very helpful when troubleshooting what you suspect to be a disk I/O bottleneck.

SELECT
   [ReadLatency] =
        CASE WHEN [num_of_reads] = 0
            THEN 0 ELSE ([io_stall_read_ms] / [num_of_reads]) END,
   [WriteLatency] =
        CASE WHEN [num_of_writes] = 0
            THEN 0 ELSE ([io_stall_write_ms] / [num_of_writes]) END,
   [Latency] =
        CASE WHEN ([num_of_reads] = 0 AND [num_of_writes] = 0)
            THEN 0 ELSE ([io_stall] / ([num_of_reads] + [num_of_writes])) END,
   [Latency Desc] = 
         CASE 
            WHEN ([num_of_reads] = 0 AND [num_of_writes] = 0) THEN 'N/A' 
            ELSE 
               CASE WHEN ([io_stall] / ([num_of_reads] + [num_of_writes])) < 2 THEN 'Excellent'
                    WHEN ([io_stall] / ([num_of_reads] + [num_of_writes])) < 6 THEN 'Very good'
                    WHEN ([io_stall] / ([num_of_reads] + [num_of_writes])) < 11 THEN 'Good'
                    WHEN ([io_stall] / ([num_of_reads] + [num_of_writes])) < 21 THEN 'Poor'
                    WHEN ([io_stall] / ([num_of_reads] + [num_of_writes])) < 101 THEN 'Bad'
                    WHEN ([io_stall] / ([num_of_reads] + [num_of_writes])) < 501 THEN 'Yikes!'
               ELSE 'YIKES!!'
               END 
         END, 
   [AvgBPerRead] =
        CASE WHEN [num_of_reads] = 0
            THEN 0 ELSE ([num_of_bytes_read] / [num_of_reads]) END,
   [AvgBPerWrite] =
        CASE WHEN [num_of_writes] = 0
            THEN 0 ELSE ([num_of_bytes_written] / [num_of_writes]) END,
   [AvgBPerTransfer] =
        CASE WHEN ([num_of_reads] = 0 AND [num_of_writes] = 0)
            THEN 0 ELSE
                (([num_of_bytes_read] + [num_of_bytes_written]) /
                ([num_of_reads] + [num_of_writes])) END,
   LEFT ([mf].[physical_name], 2) AS [Drive],
   DB_NAME ([vfs].[database_id]) AS [DB],
   [mf].[physical_name]
FROM
   sys.dm_io_virtual_file_stats (NULL,NULL) AS [vfs]
   JOIN sys.master_files AS [mf]
   ON [vfs].[database_id] = [mf].[database_id]
      AND [vfs].[file_id] = [mf].[file_id]
-- WHERE [vfs].[file_id] = 2 -- log files
ORDER BY [Latency] DESC
-- ORDER BY [ReadLatency] DESC
-- ORDER BY [WriteLatency] DESC;
GO

When I run this query against a docker-based SQL Server 2017 instance, I receive the following results.

The Latency Desc column helps to interpret the results. Latency can be classified as shown in the following table. Of course, this is a rule of thumb and your needs may vary.

Check disk latency using dbatools

Fans of dbatools.io may already know that something similar to the query above is included as part of the Test-DbaDiskSpeed command.

You can run the command as-is, only providing the instance and credentials to use. I typically add a few other options to format the output into a table and send it to a file that I can easily examine.

Test-DbaDiskSpeed -SqlInstance localhost -SqlCredential sa | Format-Table -Property Database, SizeGB, FileName, FileID, FileType, DiskLocation, Reads, AverageReadStall, ReadPerformance, Writes, AverageWriteStall, WritePerformance, 'Avg Overall Latency' | Out-String -Width 4096 |out-file c:\temp\DbaDiskSpeed.txt

Running the PowerShell script on my docker-based SQL Server 2017 instance provides the following output file.

Let’s not ask SQL Server about disk latency

Of course, there will be those who mistrust the results from either of these methods, because, well, of course SQL Server wouldn’t fess up to being the bottleneck. It’s obviously going to point the finger elsewhere.

To address those claims, check out Identify Disk I/O Performance Issues for Your SQL Server Using DiskSpd. That tool completely removes SQL Server from the discussion and objectively tests the disk I/O.

But for a quick sanity check, you can’t beat the sys.dm_io_virtual_file_stats DMV and the Test-DbaDiskSpeed command.

Looking for other ways to diagnose or improve performance? Here are a few other posts that may help.

• Should I Add Hardware Resources to My SQL Server? Do This First.
• SQL Server Performance and Windows Power Plan
• How do Views Affect SQL Server Performance?
• Is My SQL Server Configured Properly?

Want to work with The SERO Group?

Want to learn more about how SERO Group helps organizations take the guesswork out of managing their SQL Servers? It’s easy and there is no obligation. 

Schedule a call with us to get started.

The post Identifying SQL Server Disk Latency appeared first on The SERO Group.

There are a lots of great free options for learning SQL Server. I shared some in Want to Learn SQL Server? Here Are 3 Free Sources.

Many of those sources are directly from Microsoft through their labs, Channel 9, and EdX programs. Others are from members in the SQL Server community, both individuals and companies, usually in the form of training videos, user group presentations, and MeetUps.

But there are other resources, too, including some low-cost options for learning SQL Server.

PluralSight can be a great place to learn SQL Server

PluralSight has become a leading platform for training videos for all sorts of technical training. Some of the smartest people I know in the SQL Server world have top-notch training videos on PluralSight. I’m reluctant to begin mentioning names here because there are so many. I’m sure I’d accidentally omit someone. Nevertheless, stalwarts from the community who regularly share their knowledge in user group meetings, at conferences and pre-cons (in the pre-COVID days), and online in forums and Twitter’s sqlhelp hashtag have PluralSight courses. If you’re looking for one in particular, here’s a list of PluralSight authors. You can also search the PluralSight catalog of courses.

PluralSight is not free, however. You have to pay to have access to their content. They share that revenue with the contributors, so course authors get paid for their work. Note: I do not have any courses on PluralSight nor do I have a referral agreement with the company. So, I’m not hocking something to get a kickback .

For those of us that like to “try before we buy,” for the month of April 2021 PluralSight is making their content available for free. They are calling it FREEAPRIL.

Want to work with The Sero Group?

While you’re leveling up your SQL prowess, we’ll be happy to help out with any SQL Server needs you may have.

Schedule a call with us to learn more. There’s no obligation.

Footnote: I first started working with Microsoft SQL Server 6.5 back in 1996. In fact, that was one of the first certification exams I passed.

The post Learn SQL Server with PluralSight’s FreeApril appeared first on The SERO Group.

Scary things

Let’s start with scary.

Some things give us pause. We see them, think about them, and begin listing all the ways that something could go wrong.

Rappelling. Rappelling is a good example. Backing up to the edge of a rock mountain face with nothing between you and the distant ground. Leaning back as far as your can. Extending out from the safety of the ledge. Jumping backward with only a rope. It’s unnatural. Our instinct tells us we shouldn’t do it. Every fiber of our being screams “Nope!”

It’s scary.

But just because it’s scary doesn’t mean it’s unsafe. Or dangerous. With proper training and gear, rappelling is a great experience. It’s a lot of fun. And it can be done safely. We are aware of what we’re doing. We have a heightened sense and an increased awareness of the things that can wrong. And we purposely take steps to mitigate any danger present.

Scary things in SQL Server

In SQL Server, upgrades and server migrations can be scary. What happens if things go wrong? Can the migration be completed during the maintenance window? Will users be able to login when the time comes? What about performance afterward?

These are very real issues.

But, as with rappelling, we can take steps to mitigate the scary issues. We can meticulously plan the upgrade/migration. We can perform the upgrade/migration in a test environment first, allowing users to validate and approve before production is ever touched. Backups can be taken before the upgrade begins. The VM can snapshotted.

In short, we’ve identified the potential problems that could arise and we’ve taken precautions.

It may be scary, but it’s safe.

Dangerous things

Dangerous things are more concerning, especially if the danger is not recognized.

Walking along the side of a snow-covered mountain on a peaceful day. The sun warms your face. You can see for miles. Beauty abounds. It’s an exhilarating experience, a serene journey, a peaceful adventure.

Yet, it can also be dangerous. A foot or more beneath the crunch of each step, a layer of ice may have formed between two layers of snow. The ice creates a seam, a seam with little friction. It wouldn’t take much for that snow above the ice to begin sliding, creating an avalanche. And as we all know, avalanches are very dangerous for everyone nearby.

Dangerous things in SQL Server

In SQL Server, danger comes in many forms. Everything may seem ok, at least on the surface, but something could go very wrong, very quickly.

• Untested backups. Having backups but not testing them regularly.
• No alerts. A production SQL Server without configured alerts.
• Ignored alerts. Alerts that are ignored because there are too many false positives.
• Untested Disaster Recovery Plan. A crisis is not the time to try to figure out your DR.
• Assumed RPOs and RTOs. Not being able to recover to the right point in time in the allotted time.
• SQL Server health. Uncertainty about how your SQL Server is configured.
• Deferred maintenance. Preventative maintenance jobs take too long to complete so they are disabled.
• Unpatched SQL Servers. Patching is not scheduled regularly.
• Out of Support SQL Servers. There’s simply no time to upgrade so ancient versions of SQL Servers are still hanging around.
• Reacting rather than responding. Addressing the symptom rather than fixing the problem.
• Conflating HA and DR. High Availability and Disaster Recovery are similar but not interchangeable.
• Unmonitored SQL Server environments. SQL Servers are left on their own to report issues.
• Elevated permissions. Too many people with sysadmin or dbo privileges.

Of course, this is not a comprehensive list of all the dangers that can lurk in a SQL Server environment. I’m sure you have some from your experiences. Feel free to share in the comments below; I’d love to hear them.

Mitigating dangers in SQL Server

Recognition is the first step. We must identify and acknowledge the dangers inherent in our SQL Server environment. Then we can do something about them. We can take steps to mitigate the dangers, to reduce or even eliminate the problems we’ve identified.

Here are a few posts that may help.

• How Often Should I Test My SQL Server Backups?
• Is My SQL Server Configured Properly?
• SQL Server Assessments
• When Was the Last Known Good DBCC CHECKDB Integrity Check?
• Should I Upgrade SQL Servers that Are Out of Support?
• High Availability and Disaster Recovery in SQL Server
• DBA as a Service
• Securing Your SQL Servers, What Should You Audit?

And, of course, we’ll be happy to help. It’s what we do.

Want to work with The Sero Group?

Want to learn more about how SERO Group helps organizations take the guesswork out of managing their SQL Servers? It’s easy and there is no obligation. 

Schedule a call with us to get started.

(To give credit where it’s due. The idea for Scary vs Dangerous was adapted from Jim Koch’s excellent and entertaining book Quench Your Own Thirst.)

The post Scary and Dangerous Things in SQL Server appeared first on The SERO Group.

I’m reminded of a classic IBM consulting commercial from many years ago. [IBM – Throw Money At It? – Catapult Ad – YouTube]. In it, Consultant Ned advises Sir Arthur to build a giant catapult to fire the “greatest of projectiles” at the attacking sloth. If you haven’t seen it, it’s worth 30 seconds of your time.

The cost of adding resources to your SQL Server

Adding hardware may seem like a good idea. It could be a quick fix that will make users happy, at least for a while. But it’s not a panacea. It can come with a hefty price tag. So it shouldn’t be the first tool out of the bag.

Compute time

With the turn of a knob, you can add a couple of processers to the VM. A click of a mouse can increase the memory available to SQL Server. And in Azure, AWS, and other IaaS providers, faster solid-state disks are there for the asking. That’s definitely an Easy button.

However, these are not free, especially in a cloud environment. In a model where you pay by the resource and compute, these changes come with a very real and direct cost.

Even in a on-prem environment, the more resources you allocate to one VM, the fewer you’ll have available for the other VMs residing on the host.

SQL Server licenses

SQL Server licensing is, in a word, complex. So, I’m not going to get into the nuances of the different licensing models. However, SQL Server is licensed by processor. And it’s not cheap. The price for SQL Server 2019 Enterprise Edition is currently $13,748 for a 2-core pack, whereas the Standard Edition is $3,586, according to Microsoft. So adding four processors to a SQL Server Enterprise Edition system can have a memorable price tag.

Additionally SQL Server Standard Edition has some built-in resource limitations. Once those are reached, you’ll need to either upgrade to Enterprise Edition, or scale out to other SQL Server Standard Edition instances. Obviously both have some hard and soft costs.

For more information about the differences in the editions of SQL Server, see Do I Really Need SQL Server Enterprise Edition?

Before adding resources to your SQL Server

As enticing as throwing resources at your SQL Server problem may be, it’s certainly a good idea to look at other options to solve your performance problems first. In doing so, you may, indeed, find that SQL Server needs more resources. But, at least, you’ll know exactly what resources are needed, and have confidence that you are making the right decision.

So, before you turn the magical resource knob, let’s look a few areas to examine first.

Define the problem

Albert Einstein purportedly said “If given an hour to solve a problem, I would spend 55 minutes defining the problem and then five minutes solving it.”

Although there is some serious doubt around whether he actually said that, the point is still a good one. Before rushing in to solve a problem, it’s best to truly understand what you’re actually trying to accomplish. Define the problem before attempting to solve it is sound advice, even if Einstein didn’t say it.

In the SQL Server world, we’ve seen a wide array of “database problems” that weren’t actually “database” problems. So it’s a good idea to actually make sure the issue doesn’t reside somewhere else. For example, could it be a networking issue? Or perhaps a DNS issue? Maybe it’s a Citrix issue?

This is where benchmarking your systems can be of tremendous help. Have a look a the key metrics now and compare them to the metrics captured last month or last year. See if there’s a noticeable difference in response times, etc.

Clarifying the problem up front can save you time and money in the long run.

Assess the SQL Server configuration

Once you’ve determined that the problem may actually be with the SQL Server, it’s worth your time to do a high-level assessment of the configuration. This is especially important if you’ve inherited the SQL Server, didn’t set it up yourself, or it’s been a long time since you’ve looked at it.

In our SQL Server Assessments, we do a thorough analysis of the system configuration and settings. We look to see how it compares to industry best practices from a lot of perspectives, including those that would affect performance.

You may not have to go to that level of detail at this stage, though. A cursory glance at some of the more commonly misconfigured settings may prove fruitful. At a minimum, it’s worth ensuring that:

• tempdb is configured properly. Verify you have the right number, the right location, and the right sizes. See 3 Ways to Find Your SQL Server tempdb Data Files and How Many tempdb Data Files Should My SQL Server Have?
• MAXDOP is not left at the default setting. That’s seldom the best choice.
• Cost Threshold for Parallelism is not set to the default. This never the right choice.

You can also look for other areas that could cause contention, such ensuring data and log files are separated, etc.

This quick assessment may not reveal the solution, but as long as you’re looking at the SQL Server anyway, it’s a good idea to do a quick check. Here’s some more information that may be helpful as well – Is My SQL Server Configured Properly?

Also, check out our SQL Server Configuration: 5 Common Issues That May Be Putting Your Data at Risk free PDF.

Confirm maintenance plans

When a new query comes in, the SQL Server query optimizer decides how to resolve it. The optimizer considers quite a few alternatives for gathering the data. It assigns a cost to each approach and selects the plan with the lowest cost. This is known as cost-based optimization.

Cost-based optimization is predicated on having good, accurate, and up-to-date statistics. If the statistics are stale, bad decisions are made. And bad decisions lead to poor performance.

Verify that maintenance plans are in place to keep the statistics up to date. Verify that indexes are being maintained as well.

Examine SQL Server wait statistics

As SQL Server goes about its job of receiving and responding to queries, it actively manage its key resources. For example, memory, CPU, disk i/o and network throughput are all critical resources for SQL Server.

SQL Server tracks what each thread is doing, and more importantly for this discussion, what each thread is waiting on when it’s waiting. This is information is commonly called wait stats.

Best of all, SQL Server makes this information available to us. We can use the Dynamic Management Views (DMVs) to ask SQL Server what it’s waiting on.

There are a LOT of wait stats. Paul Randal has a some great information about them on the SQL Skills web site. Here’s a great place to start reading some of his detailed writing on the subject. And, of course, Glenn Berry has a ton of DMV queries.

If you’re more of a point-and-click person, you can also find some useful information in SQL Server Management Studio’s Performance Dashboard. It’s not nearly as comprehensive as going right to the DMV source, but it may provide enough information to point you in the right direction.

Identify the most resource intensive queries

Another approach consider before adding hardware resources, is to identify the most expensive or problematic queries on your SQL Server and see if you can do something about them.

The most expensive queries are not always the ones that take the longest to complete. A query for a report delivered nightly at 2:00 AM may take 10 minutes to complete. And you may not care because the SQL Server is not heavily used in the wee hours of the morning.

On the other hand, you may find a that the application login procedures take 10 seconds to complete. Each morning from 7:00 AM to 9:00 AM while users are logging in, the system is S-L-O-W! If you can get those 10 seconds down to 10 milliseconds, you’ve got huge win.

The DMVs will be your friend in searching for these. SSMS also has some standard reports built in, including:

• Top Queries by Average CPU Time
• Top Queries by Total CPU Time
• All Blocking Transactions
• Service Broker Statistics
• Top Queries by Average IO
• Top Queries by Total IO
• and much more

With “home grown” applications, those applications designed and developed by your company, you may have the freedom to make some pretty substantive changes. You can recommend database design changes and tune long running queries. You can create indexes and remove nested views. (See How do Views Affect SQL Server Performance?)

For purchased applications, on the other hand, such as electronic medical records, accounting packages, job costing applications, etc, it’s best to coordinate with the vendor. Typically, it’s not a good idea change to their application. It’ll break support and probably hinder your ability to upgrade in the future.

Other tools and resources

Here are a few other tools and resources that may help.

• Performance Monitoring and Tuning Tools
• SQLPerformance.com
• SQL Sentry
• SQL Server Performance and Windows Power Plan
• Identify Disk I/O Performance Issues for Your SQL Server Using DiskSpd
• Why 9 Out of 10 SQL Servers Aren’t Configured with Best Practices

Want to work with The Sero Group?

Want to learn more about how SERO Group helps organizations take the guesswork out of managing their SQL Servers? It’s easy and there is no obligation. 

Schedule a call with us to get started.

The post Should I Add Hardware Resources to My SQL Server? Do This First. appeared first on The SERO Group.

The Windows Power Plan Setting

Windows Server has three pre-defined power settings: Balanced, High Performance, and Power Saver. I’ll briefly summarize the settings without going into much detail. The names are pretty descriptive. (Microsoft has some nice documentation about how Windows uses and calculates energy efficiency. See Power and performance tuning for more information.)

The Balanced Power Plan

The Balanced power plan is the default Windows Power Plan setting. Unless you’ve intentionally changed the setting, that’s the scheme your Windows Server is using. It’s a good setting for many servers since it balances performance with power consumption. Typically, this setting is not good for SQL Servers, however. A reduction in processing power translates into a reduction in performance.

The High Performance Power Plan

The High Performance scheme prioritizes server performance over power conservation. All cores are unparked and all processors are running at their highest level. This is the best setting for resource intensive applications and services where throughput and responsiveness is critical.

The Power Saver Power Plan

Power Saver is exactly that. Priority is given to reducing heat, conserving energy, and lowering operating costs. This may a good setting for some workloads where latency and other performance metrics are less important. It’s hard to imagine a case when SQL Server would benefit from this setting.

Checking the Windows Power Plan on your SQL Server

A Windows Server running SQL Server should almost always use the High Performance setting. Yet, the Windows Power Plan setting is often overlooked when a new server is provisioned. In fact, we routinely find the Balanced Plan setting in place during our SQL Server Assessments.

So, to help eek out the best performance on your SQL Server, check the Windows Power Plan on your servers. Here are a few ways.

Using the Windows Power Plan Configuration

Unless you’re running Windows Server Core or Nano, you have access to the point-and-click GUI environment. To check the Power setting, open the Power Options as shown below. You can see that this server is set for High Performance.

If your server is set to one of the other plans, you can choose another plan and close the window. No reboot is required for the new setting to take affect.

Using Powercfg to check the Windows Power Setting

The point-and-click method may be fine for just a few servers. But if you want to check much more than that scripting, will be your friend. Fortunately, Windows Server has a command line utility to help – the Power Configuration (Powercfg).

Powercfg has a lot of options, including the ability to check and change the Windows Power Plan. To see the current Power Plan setting, use the command line to run the utility with the /getactivescheme option as shown below.

powercfg /getactivescheme

The current setting is printed to the screen.

If you want to change the current setting. use the /setactive option.

Using DbaTools to check the Windows Power Setting

For those of use who are PowerShell fans, dbatools.io has a great command available called Get-DbaPowerPlan. From within PowerShell, run the command and specify the server(s) you’d like to check. In the following example, I’m checking only the local server.

get-dbapowerplan -ComputerName .

You can also use the Set-DbaPowerPlan command to change the setting.

More on SQL Server performance

If your SQL Server is running with the default Balanced Power Plan, you may be running with less power than you have available. Checking and changing the Windows Power Plan can help boost the performance of your SQL Server.

Here are a few other posts on SQL Server’s performance, as well.

• Is My SQL Server Configured Properly?
• Identify Disk I/O Performance Issues for Your SQL Server Using DiskSpd
• How Many tempdb Data Files Should My SQL Server Have?
• 3 Ways to Find Your SQL Server tempdb Data Files
• Is There an Update for My SQL Server?

Want to work with The Sero Group?

Want to learn more about how SERO Group helps organizations take the guesswork out of managing their SQL Servers? It’s easy and there is no obligation. Let’s schedule a short call.

The post SQL Server Performance and Windows Power Plan appeared first on The SERO Group.