A reactive approach may get you through the day, but it doesn’t build long-term stability, security, or confidence.

What is the SQL Server Maturity Curve?

Over the years, we’ve found that every SQL Server environment naturally falls somewhere along a maturity curve. Understanding where your SQL Server environment is today and where you want it to be helps you move from firefighting to foresight. SQL Server maturity can be best understood in four stages: reactive, managed, optimized, and strategic.

Let’s look at each stage.

1. Reactive: Firefighting

At the lowest level of maturity, we have what we call the reactive stage. This is where SQL Server environments are managed more in a ‘break/fix’ mode. Something goes wrong—an outage, a performance issue, maybe even a regulatory problem—and the team jumps in to fix it. Since the problem usually catches them by surprise, they then have to spend time figuring out how to address the issue before they can start to fix it.

Banks in this stage tend to rely heavily on manual processes and have very little automation in place. There might be some monitoring, but it’s often not tailored to SQL Server and effectively too generic (maybe something like SolarWinds Orion). So, leaders don’t have a clear picture of what’s healthy, what’s risky, or what’s about to break.

A second indicator for this stage is an environment where no one person is truly accountable for SQL Server. It’s a shared responsibility, which really means no one’s watching it closely. It’s just one of many systems all lumped in together. In these environments, small problems slip through the cracks until they turn into something big.

A third indication is the assumption that the vendor or core provider is handling all necessary SQL Server maintenance. In reality, they’re not watching it nearly as closely as the bank thinks they are.

From a business standpoint, this leads to high operational costs, more regulatory findings, and frustrated employees and customers when things go down.

Most of the SQL-related budget at this stage goes toward putting out fires instead of preventing them. Unfortunately, this is still where a lot of community banks find themselves today—operating in a reactive state, vulnerable to risk, and always one incident away from disruption.

2. Managed: Gaining Control

The next stage up in the curve is the managed stage. In this stage banks start putting some structure in place.

Backups are running consistently. And I know what you might be thinking: ‘Of course they are.’ But you’d be surprised how often we hear that, only to find something very different once we dig in during an SQL Health Check.

Monitoring is usually turned on so the team gets alerted before things get out of hand, and patching is scheduled instead of done haphazardly.

You’ve probably heard the phrase people, processes, and technology. At this stage, banks are making solid progress on two of those: processes and technology. And there’s usually someone in IT who’s been given responsibility for SQL Server, though it’s often just one of the many things they take care of.

But the results are noticeable. Incidents are happening less often, performance is steadier, and compliance is easier to manage. There’s even some separation of duties starting to take shape.

Here, most of the SQL-related budget is still going toward maintenance, but now, instead of pure firefighting, a little bit of that time and money is shifting toward planning and improvement.

So the managed stage is a big step forward. Things are more stable, there are fewer surprises, and the environment is definitely safer. But it’s still not efficient—and it’s not yet resilient. That’s usually when the question shifts from ‘Are we stable?’ to ‘How can we do this better?’

3. Optimized: Running Proactively

Third is the optimized stage; things start to look and feel different. We’re no longer spending most of our time just keeping the lights on; the focus shifts from maintenance to efficiency.

Routine tasks like backups and testing the backups, patching, and monitoring are automated and standardized across the SQL Server environment. The team’s not reinventing the wheel on every server anymore. Builds are standardized and perhaps even automated.

Performance is managed proactively—indexes, queries, and resource usage are being reviewed on a regular basis. The bank finally has real visibility into capacity, performance trends, and risks over time.

And all that optimization pays off literally.

At this stage, banks start saving real money. They’re doing proactive performance tuning, right-sizing their environments, and consolidating where it makes sense. That means fewer servers, lower licensing costs, and less wasted hardware.

We worked with one client who was able to save about $2,000 a month—$24,000 a year—on just one of their Azure SQL Servers, simply by tuning and optimizing the setup.

And another bank we work with was able to cut their SQL footprint in half through consolidation and decommissioning efforts. That saves on licensing costs, management costs, etc.

But it’s not just about cost savings. This is also where security gets stronger. Misconfigurations get closed off, permissions are tightened, and the environment starts aligning with best practices like the CIS benchmarks and the principle of least privilege.

The payoff is easy to see. Customers experience faster, more reliable systems. Inside the bank, IT teams aren’t scrambling to fix the latest outage—they’re staying ahead of it. They identify and resolve issues before they impact operations or audits. SQL Server becomes a reliable foundation that actively supports business goals.

4. Strategic: Turning Data into Advantage

In the final stage, the strategic stage, SQL Server isn’t just stable or secure; it’s resilient by design.

High availability is built in. Disaster recovery plans aren’t just written; they’re tested and refined. Security is strong and consistent across the environment, and compliance isn’t something the team scrambles to prove once a year; it’s woven into daily operation.

Auditing and monitoring tools are in place. There’s clear separation of duties. And reporting infrastructure is mature enough to shift workloads where they make the most sense.

But what really sets this stage apart is how SQL Server starts to enable the business.

At this point, it’s not just about avoiding risk; it’s about driving strategy.

Data becomes a competitive advantage. Executives have access to real-time insights through analytics and reporting. They can spot trends, understand customer behavior, and make better decisions—faster.

And IT? It’s no longer seen as a cost center. It’s a business enabler—helping drive efficiency, innovation, and growth.

Moving Up the Curve

Wherever your institution is today, the goal isn’t perfection overnight. It’s steady progress. Moving even one stage up the maturity curve can dramatically reduce risk, improve audit readiness, and free up your team to focus on higher-value initiatives.

The key is to be intentional, to assess, document, and continually refine your SQL Server management practices.

Because in business, in banking, and in healthcare, SQL Server maturity isn’t just an IT milestone; it’s a business advantage.

Further Resources

• Curious where your environment stands today? We’ve created a short SQL Server Maturity Checklist to help you identify which stage your organization is in and where to focus next. It’s a quick, practical way to assess your current practices and start planning your path forward. Download the SQL Server Maturity Checklist to see where you stand and how to move from risk to advantage.

• For a deeper dive on this subject, you can watch our free, on-demand webinar, “Navigating the SQL Server Maturity Curve,” on YouTube.

Want to work with The SERO Group?

If your SQL Server environment feels more reactive than strategic, or if you’re ready to strengthen reliability, improve security, and become more audit-ready, we can help.

We specialize in helping institutions move up the SQL Server maturity curve with proven processes and a proactive approach. Let’s start a conversation about where you are today and where you want to be. Schedule a brief call with us today.

The post SQL Server Maturity Curve: How Banks Move from Reactive Risk to Strategic Advantage appeared first on The SERO Group.

Here are a few of the ways your database design can impact your application’s success:

Faster Data Retrieval

A well-designed database makes it easier and faster to find the data your application needs. For example, properly structuring your tables and relationships and having good indexes in place allows SQL Server to quickly locate the right information in your database.

Without these optimizations, your database has to do more work, leading to slower response times. Over time, as your database grows, these delays can pile up, frustrating users and affecting your app’s overall performance.

Structured for Future Growth

Applications often start small but grow in complexity and size over time. A good database design prepares your application to handle increasing data volumes and user loads. For example, splitting a large table into smaller partitions based on date ranges or geographical regions helps ensure queries are faster because they only have to search the relevant data.

As your app attracts more users, it will face increased demand for data access and processing, and your database needs to be able to handle this pressure. Techniques like indexing, query optimization, and using appropriate isolation levels play a big role in maintaining performance under heavy loads. And strategies like row-level locking or read replicas can ensure that high traffic doesn’t impact the user experience.

If these types of strategies aren’t considered during the design phase, you might face significant challenges as your application tries to scale.

More Efficient Resource Utilization

Good database design ensures that your application uses system resources like CPU, memory, and storage as effectively as possible. This means your app can perform well even with limited or less expensive hardware, saving money and avoiding unnecessary upgrades.

For example, choosing the right data types for your data columns can reduce storage requirements and speed up processing. And indexing, when done correctly, significantly improves the read performance of your queries and can cut down on memory and CPU usage, while missing indexes can slow down your entire application.

In Conclusion: SQL Server Database Design Matters

Taking the time to plan your database properly can save you headaches down the road and ensure your application is fast, reliable, and ready for anything.

Want to Work With The SERO Group?

Want to learn more about how The SERO Group helps organizations take the guesswork out of managing their SQL Servers? Schedule a no-obligation discovery call with us to get started.

The post How Proper Database Design Improves SQL Server Performance and Scalability appeared first on The SERO Group.

Just to be clear, storing sensitive information as a clear text string is a really, really, really bad idea.

Not encrypting information in a database can cause serious problems. As just one example, if the database is compromised, all user passwords could be exposed. Data breaches are becoming more and more common. If the authorities come knocking on your door, you need to be able to show them that you at least made a concerned effort to protect that data.

Encrypting text that will need to be decrypted

In some cases, you may be able to store your sensitive data as strongly encrypted text that will never need to be decrypted. For example, hashing a password used for your application login and then just comparing the hashed password for the login instead of the actual password. But, in most cases, being able to decrypt the sensitive data is going to be necessary.

In these cases,  ENCRYPTBYPASSPHRASE (available in SQL Server 2008 and up) offers one of the simplest ways for you to encrypt sensitive information in a way that can also be decrypted (by using DECRYPTBYPASSPHRASE). At its very basic, ENCRYPTBYPASSPHRASE requires two mandatory arguments: a passphrase used to generate the encryption key and the text to be encrypted.  Notice that it specifies a passphrase, not password. There is an important difference between these two.

A passphrase vs. a password

As described in the ENCRYPTBYPASSPHRASE documentation: 

A passphrase is a password that includes spaces. The advantage of using a passphrase is that it is easier to remember a meaningful phrase or sentence than to remember a comparably long string of characters.

Many people don’t realize that you can use a space as a legitimate special character in most passwords. By doing this, you can generate a much more secure password sentence (or phrase) instead of a single word. An example of a passphrase may be something like “I forgot my password!”

Just to be clear, a space is not required in your passphrase for ENCRYPTBYPASSPHRASE. If you wanted to use a GUID for your passphrase or a random string such as “Zgt9$Ex%*unZO8Z},” that is perfectly acceptable.

Using ENCRYPTBYPASSPHRASE

For the examples in this post, I am going to use the encryption passphrase “This is my Passphrase!”, and the text to be encrypted is “ABC123”.

The basic syntax is:
ENCRYPTBYPASSPHRASE(‘encryption passphrase’, ‘text to encrypt’)

There are other arguments that can be used with ENCRYPTBYPASSPHRASE (see MSDN Doc), but for this simple example we are just using the two mandatory arguments.

To view the encrypted value of the text “ABC123”, you would use this script:

SELECT ENCRYPTBYPASSPHRASE(N'This is my Passphrase!', N'ABC123');

That SELECT statement will return a VARBINARY value such as: 0x0100000093EEC20B790EF208B1FB631F0AB3028E3A8C196643C4BD578528A0DFAE7AB45B

It is important to note that the VARBINARY value returned from ENCRYPTBYPASSPHRASE is nondeterministic, meaning that even with the same input it will not generate the same output every time.  So you can run the exact same SELECT statement multiple times and get a different result each time.

Thankfully, this output has no bearing on using the DECRYPTBYPASSPHRASE function. As long as you have the correct passphrase, DECRYPTBYPASSPHRASE will successfully decrypt any of those VARBINARY results to their original value.

Storing an encrypted value in a table

Now that we know how to encrypt a sensitive text string, let’s take a look at how to store that encrypted value in a table.  Since the value returned from ENCRYPTBYPASSPHRASE is a VARBINARY data type, that is how we want to store it since this is also the data type required by DECRYPTBYPASSPHRASE.

The first thing we need to do is determine the size of our encrypted column in our table. The VARBINARY values returned by ENCRYPTBYPASSPHRASE can vary in size, with maximum size of 8,000 bytes. The size of the returned value is going to depend on the size of the actual text being encrypted. You can use the DATALENGTH function to help figure that out. If you have a way to control the maximum allowed length of the sensitive text value you want to encrypt, use that size for your table column, but try not to use VARBINARY(8000) if you don’t have to.

Here is a simple example of storing our encrypted text in the [Password] column of a table:

CREATE TABLE dbo.Users ([UserName] VARCHAR(50), [Password] VARBINARY(50))
 
INSERT INTO dbo.Users ([UserName], [Password])
VALUES ('Charlie Brown', ENCRYPTBYPASSPHRASE(N'This is my Passphrase!', N'ABC123'))
 
SELECT [UserName], [Password]
FROM dbo.Users

Using DECRYPTBYPASSPHRASE

Now that we have our sensitive text encrypted, we need to be able to decrypt it as well.  This is easily done by using the DECRYPTBYPASSPHRASE function with the same passphrase we encrypted our text string with. However, DECRYPTBYPASSPHRASE also returns a VARBINARY value, which we will have to convert to a string.  This can be done by adding a CONVERT function to our SELECT statement.

SELECT [UserName], CONVERT(NVARCHAR, DECRYPTBYPASSPHRASE(N'This is my Passphrase!', [Password]))
FROM dbo.Users

Now you should see your decrypted value returned correctly in clear text. ENCRYPTBYPASSPHRASE offers a quick and easy way for you to encrypt text in SQL Server and can be useful for encrypting sensitive information if you need to be able to decrypt it later. 

Want to work with The SERO Group?

Want to learn more about how The SERO Group helps organizations take the guesswork out of managing their SQL Servers? Schedule a no-obligation discovery call with us to get started.

The post How to Encrypt Sensitive Text in SQL Server with ENCRYPTBYPASSPHRASE appeared first on The SERO Group.

The answer is not simple.

Data roles are diverse and constantly evolving. Similarly, the lines that separate data disciplines are inherently blurred. The reality faced by active data professionals is also complex, with business and project requirements often requiring them to extend their expertise across disciplines. Consequently, data professionals often wear many hats.

Still, it’s useful for those entering the field or looking to hire a data professional to understand some of the important distinctions between data disciplines.

Here are just a few.

Common Data Roles

Data Architect

Data architects design the overall blueprint for your organization’s data environment. They define how data is stored, organized, integrated, and accessed across systems. They also ensure that your data infrastructure is scalable, secure, architected for efficient retrieval, and aligns with your long-term business goals.

Database Administrator (DBA)

DBAs are the administrators of your database systems. They possess a deep knowledge of the database engine itself, including all its native functionality and features. They are also responsible for keeping databases updated, backed up, secure, and performing optimally. DBAs also manage database upgrades and migrations, as well as database recovery in a disaster or emergency.

Data Engineer

Think of data engineers as the builders of your data infrastructure. They design, construct, and maintain the pipelines that collect, store, and process your data. Their toolkit is diverse and often includes programming languages like Python and SQL. It is also common to leverage cloud platforms like Azure, AWS, or GCP. These professionals ensure that your data is accessible, reliable, and ready for analysis.

Data Analyst / Business Intelligence Analyst

Data analysts are the storytellers of the data world. They take the data that engineers have prepared and draw out insights through creating reports, dashboards, and visualizations. Furthermore, their strong analytical skills and ability to recognize patterns allow them to turn data into knowledge. They tell stories with data to help guide decisions. Tools often include Excel and BI reporting platforms like Tableau or Power BI.

Data Scientist

Data scientists take analytics to a deeper level. They use advanced statistical techniques and machine learning to uncover hidden patterns that are often difficult for humans to detect. Similarly, the algorithms used by data scientists can predict future trends, and the outputs of their models are used to drive decision-making. Data scientists possess mathematical expertise, programming skills in languages like Python or R, and domain knowledge relevant to their industry.

Machine Learning Engineer

Machine learning engineers take the models created by data scientists and make them operational in real-world production environments. They leverage both data science and software engineering skills. ML engineers are responsible for building the systems that deploy, monitor, and scale data science models. They also manage these systems to ensure that they deliver accurate and timely predictions once deployed in a real-world context.

Tips for Choosing Your Future Data Role

Shadowing some different data professionals is a great way to get started. Depending on which role appeals to you, there are different pathways to getting started.

If you have a background in system administration, data architecture and database administration may be good avenues to investigate. Likewise, people who have enjoyed building their own home labs may also enjoy these data roles.

Similarly, if you love automation and problem-solving for technical efficiency, one of the engineering roles may be right for you. Engineers enjoy designing and building solutions for technical challenges.

Finally, if you have a mind for analytics and statistics, an analyst or data science role may be a great fit. These roles uncover the root causes of the problems under investigation in order to unlock potential solutions.

There are a variety of ways to kick-start your journey for any of these data disciplines.

1. Self-Education. There are many resources available online that can guide you through learning specialized skills. Set up a home lab environment in which to safely practice. Start to build a profile of projects and/or certifications that can showcase your new skills.
2. Bootcamps. The number and variety of bootcamps for data have increased dramatically in the last 5 years. If your schedule allows for participation in one of these intense programs, they can be a great way to upskill rapidly.
3. Formal education. If you are looking to go into a highly specialized role in a particular knowledge domain or industry, obtaining an advanced degree can be a great way to get started.

Determining which path to take will depend on what appeals to you and on the circumstances in which you are beginning your journey. The key, however, is just taking the first step.

Tips for Hiring: Which Data Role Do You Need?

Just as the path to becoming a data professional depends on individual circumstances, the type of data professional to hire will also depend on your organization’s needs and where you are in your data maturity journey.

Here are some hiring considerations for several common scenarios:

Just Starting Out

If you are building your data capabilities from the ground up, a data engineer with versatile skills is a great first hire. While partnering with system administrators and business experts, as well as potentially seeking external assistance from a DBA or data architect, a data engineer can lay the foundation for your future data work.

Business Optimization / Improved Operations

If you are looking to use your data for improved operations, a data or business intelligence analyst can work with business partners to track key metrics, identify trends, and drive data-driven decisions to find ways to improve. Likewise, by developing an understanding of the business, this person can work as a bridge between business and IT teams to help harness the full potential of your data.

Complex or Outdated Data Environment

A data architect can bring order to chaos by ensuring that your data is well-organized, accessible, and scalable as your organization grows and expands. Data architects can also be instrumental in reshaping legacy structures to meet current business requirements and make the best use of modern technologies.

Predictive Insights and Automation

Data scientists and machine learning engineers are ideal for building models that can make important predictions, optimize processes, and/or automate complex AI tasks. It is important to note, however, that having a robust, well-governed data infrastructure is a prerequisite for success for these types of initiatives.

Diverse Needs, Large-Scale Projects, or Complex Remediation

Partnering with an external data team with diverse areas of expertise can be an affordable way to: remediate complex problems, make infrastructure improvements rapidly, or design and implement large-scale solutions. Leveraging this type of support also allows you to access professionals with expertise in different data roles and may offer flexibility for scaling up or scaling down your level of support as needed.

Want to work with The SERO Group?

Want to learn more about how SERO Group helps organizations take the guesswork out of managing their SQL estate? Schedule a no-obligation discovery call with us to get started.

The post What’s in a Job Title? Understanding Changing Data Roles appeared first on The SERO Group.

This interpretation misses a key benefit of data governance, however: competitive advantage. We cannot diminish the critical importance of regulatory compliance and its role in shaping governance efforts, but it’s important to understand that this is not the whole picture when it comes to data governance.

Governing Data as an Asset

Data is a business asset, not unlike cash. If you asked any business leader whether their cash flow processes needed controls to ensure their accuracy, integrity, and protection, they would likely give you a quizzical look and assume you were baiting them into some trap…of course their cash flow processes need controls.

Similarly, data has the potential to guide and inform nearly every aspect of a business. To leverage data effectively, businesses must ensure its integrity in an ongoing and reliable way. From financial reporting to performance evaluation to operational efficiency, data quality sits at the heart of modern businesses. Later, we will see some examples of how data quality and integrity issues can have significant impacts on SMBs.

Additionally, company data assets need to be secure and protected—arguably even more so than other company assets. This is because many personal data elements do not ultimately belong to a business. These considerations impact regulatory compliance and are essential for effectively and responsibly leveraging data as the valuable asset that it is.

The policies and processes that ensure the availability, usability, integrity, and security of data assets are what we mean by data governance.

Did you know that…

• 97% of data leaders state that their companies have experienced the costs of disregarding data quality and integrity in the form of lost revenue opportunities, inaccurate performance forecasting, and/or poor investments.
• More than 87% of small and mid-sized businesses (SMBs) collect or process sensitive customer data that could be compromised.
• Small businesses spend an average of $955,429 to restore normal business operations in the wake of successful attacks.

AND…

• About 85% of data science and analytics projects fail due in part to disregarding data governance processes.

Four Common Challenges for SMBs

Challenge #1: No Buy-In

To effect change within an organization, you must secure buy-in from key constituents. For data governance, these important players begin with executive leadership and extend through the organizational hierarchy to front-line staff. However, some SMBs question whether data governance has any real applicability for them. In response, consider: Do you store sensitive customer, client, or patient data? Do you keep sales, productivity, performance, or program data for decision-making? If you answered yes to any of those questions, data governance principles do have applicability in your environment.

Business leaders and decision-makers should prioritize data governance because their decisions directly impact the company’s value and profitability. Without data governance policies, decision-makers can only hope that their data-driven decisions are well-founded. Inaccurate data can lead to poor decisions in ways that are difficult to detect or explain even after the fact, leaving decision-makers accountable for the outcome.

Securing employee buy-in is equally important when implementing data governance. Buy-in should stem from an understanding of the shared responsibility for the data that drives business outcomes. A collective sense of accountability for data quality works to prevent resentment and misunderstandings towards your policies that could undermine even the best data governance efforts.

Tips for Success:

• Elicit interest and assistance from vested stakeholders. Leverage opportunities to ease business pain points through data governance. I’ll list some examples in “Data Governance in the Wild” below.
• Pave a step-by-step path to mature your data culture. Start with education, delegated responsibility, and distributed data ownership.
• Align policies with the existing strategic objectives of the company. Be specific about the expected business outcomes of your efforts.
• Empower business stakeholders to take ownership of and make decisions about business data.
• Create strategic goals and an initial roadmap. This will work best if it is an ongoing organizational effort that is largely led by the business, not by IT.    

Challenge #2: No Budget

Chances are that unless improving data governance is already a top business priority, perhaps because of a recent security incident, audit, or a particularly problematic data irregularity, you will be up against budget constraints when introducing formalized data governance to an SMB.

This is a challenge, but not a barrier. Yes, there are tools on the market that can be very helpful, but they are by no means necessary for SMBs—and certainly not at the outset.

Tips for Success:

• Start small and go slow. Introduce change gradually by beginning with what you can accomplish internally. Define and document business logic for your most critical data objects, and then introduce business processes to enforce that logic.
• Treat data governance as a process of continuous improvement rather than a costly one-time project. Small, inexpensive wins can produce significant results.
• Avoid a box-checking mentality. Use internal data audits to showcase regulatory compliance, but don’t let minimal standards dictate strategy. Prioritize business advantage over compliance, focusing first on areas with immediate value, such as labor cost savings, enhanced reporting, or boosted analytical capabilities.
• Leverage tools strategically. Develop your strategy and framework first. Look to tools second, and only for ease of implementation and execution. Tools alone will not provide true data governance.

Challenge #3: No Champion

You know that data governance is important for your organization and that someone needs to advocate for it. You also might not have the authority, capacity, or desire to be that person. In that case, use the suggestions about securing buy-in from Challenge #1 to find or create a data governance champion within your organization.

Tips for Success:

• Business and IT need to collaborate for effective data governance. There should be a designated senior IT team leader who is collaborating closely with at least one senior business leader. In a small organization, this could simply mean your one IT resource working with the business owner. Regardless of the size of the organization, the effort should not be one-sided.
• Champions should not limit their scope to policies alone. Having centralized documentation, tools, and processes at the ready will help to avoid individuals and teams developing their own tools and processes to implement the policies their own way. Disparate processes can lead to inconsistency and diminished success.
• Champions need to be committed and persistent. Don’t roll out data governance processes and forget about them. Remember that education and training will need to be ongoing, as will the refinement of governance practices to keep pace with technology and changing business requirements.
• Champions should consider the needs of the employees that work with the data when creating policies and approving tools. If your governance initiatives are seen as an unnecessary burden, workarounds will be developed that could compromise your efforts.

Challenge #4: Internal Skills Gap

You have the needed organizational buy-in, a small budget, and even a champion, but your IT team is small (or outsourced), without any data professionals on staff. Where can you begin?

Remember, the key is to go slowly and implement organically. The full implementation of data governance will ultimately require technology expertise, but defining rules, requirements, and workflows may not. Start there, then reach out for help with technical implementation when you’re ready.

Tips for Success:

• Research any compliance regulations applicable to your business sector. Perform an analysis of how your business is meeting these requirements. Where you find gaps, assess a strategy for how to fill those gaps. Implementation can be left to a third party as needed, but be sure that you understand what needs to be done. If you are unsure about how to get started, use some of that budget to reach out for support.
• Perform a security audit of your data systems. Adjust and delete credentials and permissions as appropriate. Review third-party system access. Inventory inbound, outbound, and internal data pipelines and evaluate for best practices. Then set up ongoing processes and/or reports to assist with security monitoring.
• Look for critical data objects that need to be formally defined. Beginning with mission-critical data elements, document all business logic and metadata requirements associated with these elements in a data dictionary or catalog. This doesn’t need to be elaborate, but it should cover all key data elements. Start in one business area and expand out from there.
• Assign ownership and accountability. Look for data owners who are invested in the quality of the data within their domain because of their connection to the processes that produce or consume the data.
• Upskill and/or seek third-party guidance as required. Provide access to resources for interested staff to get trained to become data owners, stewards, and partners in data governance. Reach out to third parties for guidance in areas that are outside the internal expertise of the organization.

Data Governance Challenges in the Wild

Below are some real challenges faced by SMBs that could be solved with improved data governance processes.

Local Restaurant Chain

A small, local restaurant chain encountered a discrepancy between a company report on drink sales and individual location reports generated in Excel. This irregularity surfaced after a competition to incentivize staff to sell more drinks by holding a competition between locations with a cash prize. At the end of the competition, when managers compared their individual reports, Store A had sold the most drinks. When using the central company report, Store B had. Which was correct?

Upon investigation, one of the restaurant locations had long ago coded milkshakes as drinks. All of the other locations coded them as desserts. This was never audited or corrected. As a result, the individual reports that were rolled up by individual menu item codes produced a different result than the central report that was rolled up by the “Drink” category. This confusion affected competition results as well as drink and dessert sales analytics and other location-to-location comparisons.

Small Food Manufacturer

A large grocery store chain approached a regional manufacturer of salad dressings and condiments, requesting the production of one of its products as a generic for the chain. Similarly, a bulk food retailer asked the same manufacturer to produce a bulk version of the same product. The contracts were accepted, but the original product, the generic version, and the bulk version of the product are all assigned different product codes without being rolled up to any parent product category. When the accounting office ran their standard report of overall sales trends by product, they had no way of recognizing that these three items represented the same underlying product and ended up producing a skewed and incomplete product analysis.

International Boutique Wholesaler

A private international wholesaler with dispersed brick-and-mortar locations was having continual problems with their inventory. Customers complained that the website frequently showed products as available when they were not. Retail associates complained that the inventory in the POS was incorrect and frequently showed negative inventory when stock was actually present in the stores.

One of the issues in this scenario was that individual store inventory was manually counted and entered with different workflows, lacking a centralized mechanism for tracking inventory. Furthermore, associates often used the “miscellaneous” category to sell products that were at the stores but didn’t have a corresponding inventory item recognized in the system. These issues produced a myriad of operational, financial, and analytic discrepancies, as well as increased costs in the form of labor inefficiencies, lost revenue, and misplaced or stolen inventory.

Social Services Organization

A social services organization gathers information about its clients to produce an annual statistical analysis of clients and outcomes. This nationally distributed, highly regarded report serves as a source for scholarly research and influences funding for numerous public and private institutions.

However, the organization faces challenges in bringing consistency to its intake processes. Since many clients arrive in crisis, intake steps that can be skipped by staff often are because intake personnel are in a rush and do not realize the importance of each step. Furthermore, the application being used to collect this valuable information does not require answers to many questions that are deemed critical for this organization and the annual report. The organization has attempted many process changes to improve the consistency of this workflow. However, the problem still persists and is consistently cited as a limitation of their statistical findings.

Is data governance important for SMBs?

This is just a small handful of examples of how a lack of data governance can affect SMBs. In fact, once you’re on the lookout for data governance issues, you may start to notice them everywhere.

Additionally, we have only begun to touch on the security and compliance aspects of data governance. Many of the challenges to implementation outlined here are less prominent when a regulatory requirement is at play since compliance is mandated. Even when regulations play a less obvious role, most of us are aware of the gravity of security risks. It is important to note, however, that this security risk is even greater for SMBs than for the largest companies. According to Veeam’s 2023 Data Protection Trends Report, 85% of ransomware attacks targeted small businesses.

The take-away? If your business or organization retains data, and very few (if any) businesses do not, formalizing some level of data governance will help you to securely and responsibly leverage that asset to drive your business.

Further Reading

• • More interesting statistics

• • A privacy audit checklist

• • A guide to data governance for small businesses

• • Data governance frameworks

• • Amazon’s guide to data governance for SMBs

Want to continue the conversation about data governance for SMBs?

To learn more, you can click here to access a recording of our May 14, 2024 webinar on Data Governance for SMBs.

Need help? We can help you tailor practical data governance solutions to meet the specific needs of your SMB. Schedule a call or send us an email. 

The post Data Governance in Action: 4 Challenges for Small and Mid-Sized Businesses (SMBs) appeared first on The SERO Group.

In the last post, we talked about applying the Kaizen approach to data management to achieve a culture of continuous improvement on our data teams.

In this post, we will use the KonMari method of simplification, recently made famous by Marie Kondo, as a lens for considering what to keep and what to purge in our business data repositories.

As always, we will also give some best practices for how to establish policies around archiving and purging your company data.

Data Lifecycle Management: Archiving and Deletion

As mentioned in previous posts, the final phases of Data Lifecycle Management are Archiving and Deletion.

These phases help to ensure that we keep and maintain only that data which is required for our business. But how do we determine what to keep active, what to archive, and what to purge?

We can apply some concepts from the KonMari simplification method to our data strategy here to help us decide.

Rule 1: Make the Commitment

Kondo states that the first step in KonMari is committing to achieving your goal. This may seem like an obvious first step for any endeavor, but many companies fail to establish a data retention strategy.

It is not uncommon for businesses to take the approach that keeping all data (sometimes even in a “hot”, or readily accessible, repository) is the way to go. This tactic usually stems from either:

1. an explicit belief that you cannot go wrong with keeping too much historical data
   or
2. from having no capacity to prioritize a retention strategy.

Either way, the “keep everything” strategy is misguided for 3 reasons.

First, the more data you keep, the more time it will take to recover in the event of a crisis.

Crisis can take the form of:

• a lawsuit or an audit that requires retrieval of specific information

or

• a natural disaster, human error, criminal activity or another event that demands restoration of data to a particular point in time.

In any case, restoration time is critical during these events. The more time it takes to retrieve the required data from your archive, the longer it will take for the business to recover.

Second, while data retention is a necessity, it is also a liability and entails responsibility.

Businesses must take the responsibility to respect consumer privacy rights very seriously. Part of this responsibility entails keeping consumer data for no longer than is required or for any purpose other than that for which the consumer gave consent. Even if your company does not fall under the regulatory jurisdiction of privacy laws like GDPR, CCPA, or PIPEDA, the business is nevertheless liable for securely and responsibly maintaining its consumer data.

With any data that is retained comes the possibility that it could be stolen, leaked, or misused. This risk is unavoidable, but preserving unnecessary archives of historical data is a liability that ought to be avoided.

Third, “Data stores don’t grow on trees…”

A well-crafted data strategy can reduce the financial cost of maintaining your data repositories, but increasingly large data stores cost the business money nevertheless.

There are also performance, time, system resource, and opportunity costs associated with maintaining large data stores.

So, in short – make the commitment to tidy up your unwieldy data repositories!

Rule 2: Imagine Your Ideal

Compliance regulations do some of the work of envisioning the ideal for us in the data world. Still, take the time to consider the ideal composition of your data repositories. Doing this can help you to think strategically about what should be kept, how and where to keep it, and for how long.

As mentioned in the previous post in this series, consider both regulatory requirements and the needs of the business for reporting, analytics, and strategic planning when determining what to keep. Consult business leaders and business analysts about what data is needed and for how long. You can even create a formal data contract for critical data elements in your business.

Rule 3: Finish Discarding First

Obviously, deleting data should always be done with extreme caution and forethought.

Nevertheless, once you have performed an audit of your data repositories and have determined your retention strategy, you should begin implementation by purging unnecessary data. We will discuss more about how to perform this action safely and according to best practices below.  

Rules 4 and 5: Progress by Category and in the Right Order

For the purposes of purging and archiving data, we should be thinking in a criticality/age matrix like the one below – beginning in the upper left corner and working down and to the right.

You should make incremental passes across departments in these stages, beginning with the oldest and least important data in each departmental area.

Rule 6: “Does it Spark Joy?”

Ok, ok – I admit this iconic final question from Kondo is much less suitable to data retention. However, there still may be an important (if less-than-perfectly-measurable) question that should be asked by data teams before coding a delete.

Does a business leader strongly prefer to retain certain data despite the lack of any clear regulatory or business-driven reason for doing so?

If so, keep it…unless doing so presents a serious risk or concern. If you feel there is a serious risk, continue to voice your concerns. Otherwise, just wait and continue to get clarification.

Now for some best practices…

Keep in mind that the practices listed here do not include the critical practice of backing up your production data, which has been discussed in previous posts (here, here, and here). Always make sure that backups are in place before beginning to archive or delete data.

1.     Replicate and archive data “in-flight”.

Archiving and/or replicating your data at various points in your pipeline is a best practice. Process failures in data pipelines are not uncommon, and you need to be able to recover data from earlier stages of the pipeline if you need to reprocess the data.

3 common examples of this practice, which should include purging data after an established retention period, are:

• Moving imported files to an archive folder
• Replicating transactional databases to a staging database before further processing by downstream systems
• Staging imported API data in its own table or database before integrating with internal systems.

2.     Archive in cold storage and protect the archive.

Consider how you will store data that has served its immediate purpose and has been determined to be a candidate for long-term storage. There are pros and cons to each method, and a combination of archiving methods may be appropriate for your different data sets. Here are some options and considerations.

• Onsite physical storage
  • Pros: ease of access in an emergency, familiar technology
  • Cons: vulnerable to tampering, theft, and physical damage/natural disasters
• Offsite storage (tape, optical disk, magnetic hard drives)
  • Pros: well-established, relatively inexpensive, usually more secure than onsite
  • Cons: slower recovery times
• Cold cloud storage
  • Pros: speed and ease of recovery, alleviated burden of maintenance, built-in security, inexpensive
  • Cons: potentially less familiar to established IT departments than traditional methods
• Data lake
  • Pros: accessibility, speed and ease of recovery, suitable for use with emerging technologies, built-in security
  • Cons: less-established, steeper learning curve, potentially expensive, requires careful governance of user access

3.     Don’t forget critical data that is managed by third parties.

Over time, businesses may find that a significant body of company data resides in data stores managed by third parties. Since maintenance has often been delegated to vendors in these cases, archiving this data is sometimes overlooked.

While there may be an ability to set a retention policy, it can be very beneficial for many reasons to work with your vendors to set up an extraction process to archive a copy of your data from these systems into your own repositories as well.

4.     Establish retention schedules and procedures for purging data.

Business, regulatory, and legislative needs dictate what should be saved and for how long, and these may differ between data sets. Establishment of policies and procedures for deleting data will ultimately be the responsibility of data owners.

These policies should address the following areas:

• Who is authorized to purge data?
• In what maintenance windows can this process occur since deletion can go slowly and data processing jobs/replication must be disabled?
• How will notification be given to the business?
• What validation and integrity checks must be in place?
• What rollback procedures will be used if necessary?

Just the tip of the iceberg…

There is much, much more to say about all these topics.

If you have made it as far as committing to cleaning up your data but the rest seems overwhelming, never fear! There are many vendors that are happy to help with all levels of assistance.

If you have a good handle on your archiving and deletion processes but would like assistance with a SQL Server implementation of them, reach out! We are here to help.

The post Archiving and Deletion Strategy…KonMari for Data Management? appeared first on The SERO Group.

This refrain is more common than you think in IT departments of all sizes. Or maybe you live that reality every day and are fully aware that clunky, error-laden processes eat away at your team’s efficiency (and morale).

Data management and continuous improvement may sound like they should always go together, but they often don’t.

Many times our data management practices involve too many business-critical data processes that break regularly and need to be improved, but we have no time to make the needed improvements because there are so many data processes that need to be “managed” (a.k.a. remediated regularly).

So, how can we break this cycle?

Welcome back to the second of three posts on how to refine your strategy for Data Lifecycle Management (DLM)!

In this post, we will focus on Data Management as the second of the three DLM stages: Data Collection, Data Management, and Data Deletion.

Kaizen for Data Management

The Kaizen approach, famously championed by the Toyota corporation, suggests that small organizational changes can lead to a culture of continuous improvement. This culture will ultimately lead to better processes, greater efficiency, improved outcomes, and increased morale.

The Kaizen Institute states,

“As part of the corporate culture, continuous improvement becomes an ongoing process integrated into the organization’s daily activities. Employees are encouraged to challenge the status quo, suggest ideas, and implement improvements. Continuous learning and development are valued, and mistakes are seen as growth opportunities.”

This means that adopting a Kaizen approach to your data management strategy can be a lever for driving a continuous improvement culture on your data team without sweeping, drastic changes.

Small improvements to existing processes can slowly bring significant reduction in process failures, and improvements in efficiency, accuracy, and team morale. (Here is a short article about applying a Kaizen approach in an IT context.)

So, how and where can we improve our data management?

Where should you look to start identifying small improvements that might be implemented?

Consider the areas below with your team. Most likely you will find that you are very strong in some areas, but perhaps there are areas that have not been addressed at all. Start with the lowest hanging fruit, and bit by bit you will find that you are slowly filling the gaps and addressing the technical debt that every established data team faces.  

First, analyze your data structures.

The applications, tools, and data processes in place for your company will impact the data structure that needs to be in place for it to be usable. Unfortunately, these requirements rarely align.

When you think about the flow of your data, think about consistency of format and type. As data flows into your system, it is often riddled with discrepancies in format, data type, and even the information it contains (but we will save that for another post).

As your data flows downstream toward the consumer, it should become more and more aligned in these areas. Why? Because the more points of contact that your technical teams must have with it (to transform it for particular use cases, etc.), the more points of failure you can have.

Strategic policies and governance and centralized data management can really help, but you don’t need an operational overhaul to improve!

In line with a Kaizen approach, try encouraging small changes in these areas:

Establish data standards

This will be an ongoing process. You will want to give thought to what your core standards should be, especially for mission critical data elements like identifiers, account numbers, etc., since these are more difficult to change once processes are mature. However, your standards will expand and refine as your business matures its data processes.

Adopt an enterprise modeling tool

Document and catalog your data standards using a modeling tool. Include all the metadata associated with your data objects and their relationships. The business will use the resulting documentation at every level (system administration, development, business analysis, and report consumption) for understanding and interpreting the data.

Transform your data with consistency

Wherever your transformation layer lives in your processes (and hopefully there are as few of these as possible), always architect toward your established data standards.

Establishing governance and centralized management can really help here, but feel free to start small! Apply these principles to new processes and only to established processes as they require other changes. Encourage a culture that celebrates these improvements and looks for opportunities to make things better.

Implement database source control

That’s right – employ a source control process for your database objects. Many companies do not take this step. However, having source control in place does not only protect your team from losing important data objects. It can also help ensure that new structures follow established standards when code reviews, pull request approvals, and other best practices are in place.

Structure your deployment process

Lastly, establish protocols around deployment. Some options include:

• Creating a deployment cadence that uses established deployment windows
• Setting up a change advisory board for reviewing changes before approving them to be deployed to production
• Designating deployment managers that are responsible for deploying code
• And, of course, you can always automate your deployments! Just be careful to include the appropriate guardrails.

Remember – slow and steady wins the race with continuous improvement.

Second, evaluate your data pipelines.

Outside of data structure, there are other data process considerations that need to be evaluated as well.

Accuracy & Reliability

• Are your data ingestion and replication processes accurate and reliable?

Sometimes when evaluating our pipelines, we find that issues with error handling, purge processes, SFTP, APIs, replication, logging or any number of other processes are causing duplicative, inaccurate, incomplete, or undelivered data transfers. Look out for these and correct them as you find them.

Maintenance & Scalability

Also, ask yourself these questions:

• Are you frequently stretching the limits of any of your allocated hardware, VMs, databases, or network resources?
• Are any of your system resources in need of upgrades or patching? Are you missing protocols to ensure that these are completed?
• Are there other systems, applications, technologies, or vendors that might suit your current or projected needs better?
• Are your data processes too slow? Do they struggle with the amount of data that must be processed by them?

If your answer to any of these questions is “yes”, then you have opportunities for improvement (slow and steady…).

Third, never forget about security with data management.

Security should always be top of mind when considering your company data. Here are some areas to evaluate.

Security – External

Review the security around the infrastructure supporting your company’s data processes for points of external connection. Pay particular attention to any processes that utilize third party tools or that export or extract data to/from external sources.

Security – Internal   

For internal sharing and usage, security measures should be concerned with careful provisioning of access to data and systems. For lower-level systems, be sure to mask or de-identify any sensitive data.

Further, for sensitive or confidential data, give careful consideration to protecting against any intentional or unintentional data leaks. Areas to consider creating policies around include:

• Unsecured physical devices or paperwork
• Keeping only what data is necessary
• Emailing sensitive data
• Downloading data to personal devices
• What to do if a suspected data breach has occurred

Is your head spinning? Don’t worry!

Remember that data management is an ongoing process of continuous improvement, and we will delve into many of these topics more deeply in upcoming posts.

In the meantime, if you have a pressing need and could use some help detailing a roadmap, let us know! We love to help empower continuous improvement with our clients.

“We cannot become what we want to be by remaining what we are.”

– Max DePree

The post Data Management Strategy: A Kaizen Approach appeared first on The SERO Group.

Welcome to the first of three posts on how to refine your strategy for data lifecycle management. In this post, we will look at how to evaluate your data collection processes for improvements.

Data Collection in Data LifeCycle Management (DLM)

As has been noted in a previous post on the difference between Data Lifecycle Management (DLM) and Information Lifecycle Management(ILM), there are fundamentally 3 phases of Data Lifecycle Management (DLM) into which all physical data-related tasks fall: Data Collection & Creation, Data Management, and Data Deletion.

As an IT leader, there are two important exercises you should perform to evaluate your data collection strategy. By performing these, you will produce living documents that should guide how your company creates, ingests, and consumes data now and in the future.

First, perform a data collection audit.

The first step in evaluating your DLM processes is to gain a complete understanding of the data collection processes your company is currently using. The best way to do this is through an internal audit.

Your data collection audit should include answers to the following questions:

• Where is data coming into your systems (websites, transactional systems, vendors)?
• What systems or processes are used to create or collect data (software, web forms, APIs, FTP)?
• What data formats are being leveraged by these processes (SQL, JSON, CSV, XML)?
• What security, threat mitigation, backups, and archiving processes are in place for these processes and data stores?  (Here is a good summary of what to look for.)

Finally, examine the information you gathered through a strategic lens. Look for vulnerabilities, inefficiencies, and pain points in your processes. Then work with your team to devise a strategic plan and implementation timeline for achieving improvements in these areas.

Second, create a data tracking plan.

Now that you have audited your data collection processes, you should give thought to why you are collecting the data that you are and what data needs to be collected. Consult business analysts in your company about which metrics they would like to track. This will help you understand what data points need to be collected. Likewise, find out what government regulations dictate about what data should be collected and retained.

Ask questions like:

• Is the right data being collected?
• Are there any missing data points?
• Are you collecting irrelevant or duplicated data?

Bridge the Gap

Undoubtedly, it is tricky to bridge the gap between the business, which has ideas about what data they would like to track, and the technical team, who knows how to track it. A data tracking plan is a tool that can help with this.

While some people strictly define what a data tracking plan must consist of, a simple plan is often sufficient. Your data tracking plan defines your primary business objects (customers, products, stores, etc.) and the metrics or events surrounding them that your business would like to have more information about.

Before spending time creating your own, take a look at the many templates available to get you started. Here is a link to an evaluation of a few free templates to start your research.

Create the plan

Once you have your template, start your internal planning discussions with questions like:

• What core business objects are we concerned with?
• What metrics do we care about for those objects (that is, what do we want to track about them)?
• Why do we want to track these metrics?
• What data needs to be collected to obtain these metrics and how will it be defined?
• Where can the data be obtained? Do we already collect it?
• Who will govern the information once we have it?
• Who will manage the data collection?
• What format does the data need to be in to be useful?

It can certainly be challenging on many fronts for the business and IT to come together to create a data tracking plan. However, facilitating this will be well worth the effort in terms of the clear data strategy objectives that will be produced. Avoiding the costs associated with misguided data projects will more than outweigh the time and energy spent in coordinated planning.

Finally, update your data strategy, and implement changes.

Once you have assessed your data collection processes and have identified improvements, you’ll need to assign priorities to your findings. Work with both the business and your technical team to set these priorities, as well as to build a roadmap for implementation.

As can be seen, the information you have gathered through cross-functional cooperation and through using these tools will help you to make a strong case to business leaders for the importance of these strategic improvements.

Want to learn more?

Looking for more information about Data Strategy and how it can help align IT and business goals? Check out these posts.

If you’d like to learn more about how we approach Data Strategy, or if you have some concerns about your SQL estate, give us a call. We can help.

The post Data Collection: Two Key Tools to Improve Your Data Strategy appeared first on The SERO Group.

Recently, we began a series of posts on data strategy by asking whether your company data is an asset or a utility. We then uncovered 10 Data Storage Considerations to Improve Your Company Data Strategy. In this post, I will argue that recognizing the distinction between DLM and ILM is important for IT leaders to help their companies get the most out of their data.

Now, the difference between Data Lifecycle Management (DLM) and Information Lifecycle Management (ILM) might initially seem like hair-splitting. However, while DLM incorporates management policies for the physical aspects of data as data (type, size, location, age, etc.), ILM addresses the content of the data as information (its accuracy, reliability, sensitivity/confidentiality, etc.).

Here are some of the differences and why both are important to consider.

Data Lifecycle Management

At its highest level, DLM addresses data creation, data management, and data deletion.

Within these categories, there are many other elements of data management that are included as well:

• Creation / Collection
• Classification
• Redundancy
• Duplication
• Integrity
• Usage and Availability
• Sharing
• Storage
• Security
• Archiving

All these data process elements when viewed through DLM are fundamentally concerned with the physical aspects of managing data. Failing to address these areas in your data strategy is to gamble with the health of your data structures and the value of your company data.

Information Lifecycle Management

On the other hand, ILM addresses issues associated with the information the data contains. ILM establishes policies that manage the data quality, business relevance, regulatory compliance, and legal liability of the data.

Many of the specific areas addressed by ILM establish protocols for processing data in a way that ensures data accuracy, reliable delivery, protection of sensitive information, and compliance with data privacy laws. Some elements of ILM can include:

• Data De-Identification / Masking
• Data Quality Frameworks & Audits
• Development and QA Environment Refreshes
• Source Control
• Master Data Management
• Classification and Governance
• Sharing & Usage
• Security
• Regulatory Compliance Audits

While many elements of the data lifecycle have relevance in both management models, they are viewed through a different lens in each model.

So…Who Cares?

So, why does recognizing this distinction matter? Will your business really suffer without it?

The answer is that it matters because if this distinction is not recognized, the two sets of management policies can get conflated, with one getting largely ignored. When either DLM or ILM is neglected, important elements of the data lifecycle can appear to be fully managed when in fact they are only partially addressed.

Perhaps even more importantly, when IT leaders do not see these management policies as distinct, they risk missing out on the benefits that come with using both together, like:

• Improved system performance
• Increased availability and accessibility of data
• Improved data quality
• Increased data consistency across the organization
• Improved recoverability
• Increased security
• Increased user satisfaction
• Controlled costs
• Improved regulatory compliance

In upcoming posts, we will unpack these models further while exploring more ways to improve your data strategy.

Want to work with The SERO Group?

Would you like some outside input in any of these areas? We love to work alongside IT leaders and their teams to help them establish the use of best practices in their data environments.

If that’s something you’d like to learn more about, let’s have a conversation.

The post Are Information and Data Lifecycle Management Processes Different…and Who Cares? appeared first on The SERO Group.

Recently, we began a series of posts on data strategy by asking whether your company data is an asset or a utility. In this post, we will look at 10 key data storage considerations that should be evaluated as part of your company data strategy.

What is Data Storage?

Data storage can be broadly defined as the collection and retention of information using digital technology.

Sometimes data storage refers to the physical hardware itself. Other times, it includes the processes connected with storing data. Both are important when considering data storage in the context of your company data strategy.

So, we can think about data storage in two categories. First, we will consider the storage infrastructure itself, including both the hardware and the data management systems. Second, we will touch on managing that platform – including backup, archiving, and retention policies.

Data Storage Platform

The data storage technology and vendor you choose will be a critical part of implementing your data strategy. Here are 7 important areas to evaluate.

1) Size and Growth

• How much data does your company currently store?
• How rapidly is new data generated?
• How fast and how large do you expect your data stores to grow?

Awareness of the current size and growth patterns of your company’s data ecosystem is crucial for accurate planning.

2) Data Storage Infrastructure

• What does your current IT infrastructure consist of?
• Have you recently made investments in updating hardware, or is company equipment nearing end-of-life?
• Do you have internal talent that is tasked with managing that infrastructure?

Any technology strategy must ask: What can and what should we manage internally? This question applies to data storage as much as to any other part of IT.

3) Data Types and Workflows

• Is your company primarily concerned with transactional data?
• Do you have large bodies of reporting data?
• What about unstructured data?
• Does company leadership have plans to expand the ways it uses or generates data in the future?

Different technologies support certain types of workflows better than others. So, knowing what your company wants to get out of its data both now and in the future will impact your data storage decisions.

4) Traffic

• How many data consumers do you expect to have and how will they access the data?
• How many concurrent users or connected devices do you anticipate?
• What types of data consumption are typical?

How and how much your data is consumed will also impact data storage requirements. Therefore, accurately projecting this usage can save countless headaches later on.

5) Data Storage: Cloud vs. On Prem

Evaluating whether to use a cloud-based or on premise data storage solution is critical to your data strategy. So, be sure to include all of these areas when performing your analysis:

• Security
• Budget/Cost
• Regulatory Compliance
• Location/Distribution

Do your homework. Prominent tech leaders have strong arguments on both sides of the cloud vs. on prem debate. So, what is right for your company may not be the same as what is right for your industry peer. Evaluate the options in the light of your company’s budget, needs, and current state to come to the best decision.

6) Data Storage Vendor

Choosing the vendor of your solution is your next big decision.  For a good article about questions to ask when evaluating vendors, see this article.

While we at The SERO Group specialize in SQL Server and Azure SQL, there are many vendors for data storage. Which vendor is right for your company? Budget, security, the requirements of the business, and the current state of your data environment will likely be your biggest considerations.

7) Licensing for Data Storage

Navigating licensing requirements can be a pain. However, carefully researching features that are unlocked with each licensing tier is important to getting the best service and support from your chosen data storage solution.

A careful cost-benefit analysis of licensing can be a game-changer for unlocking the highest potential of your data team as well. Sometimes advanced features are rightly deemed superfluous for your business case. However, at other times, higher licensing tiers afford security, recovery, performance, efficiency, or scalability options that are well worth the cost.

For SQL Server (whether on prem or on an Azure VM), this means considering Standard vs. Enterprise edition. Managed instances and Azure SQL databases have other important licensing considerations. See here for Microsoft’s description of the differences and here for a discussion of the differences between some of these options.

Management Considerations: Backups, Archiving, and Retention

8) Backups

• How are you currently backing up your data?
• Have you tested the recoverability of your backups?
• Would your backups still be available if a site-level disaster occurred?

Your backup strategy is critically important when thinking about data storage. Disasters and crisis events happen. In fact, we at The SERO Group have just about seen it all in terms of recovery efforts. Depending on your data storage platform, different backup considerations will apply to you.

There are usually three levels of backups that could be relevant to think about: local, image-level, and offsite. So, be thorough when creating your strategy. Remember – frequency and redundancy are keys to a strong backup strategy. Your future self will thank you if confronted with a crisis!

9) Data Archiving / Table-Level Snapshots

So, why archive or take table-level snapshots of data if you have a solid backup strategy in place?

While this may not always be necessary, taking a snapshot of point-in-time data can be helpful or necessary for important data that may fluctuate (financial data, for example). These snapshots can also act as a point of redundancy in your backup strategy for critical data.

However, archives can require a significant amount of storage. So, how and where to store large amounts of snapshot data should be evaluated if an archiving strategy is used.

10) Data Retention

• How much data do you need to keep and for how long?
• How long should your backups and archived data be in short-term storage?
• What about long-term storage?

You will need to consider any regulatory requirements that apply to your data when answering these questions, as well as the operational needs of the business.

Want to work with The SERO Group?

Data storage can be a challenge, especially for growing companies. How to store the data? When to archive it? Where to store it? Who is responsible for it? All can be be challenging questions.

Our clients depend on their data. They need to know that their data is protected, that it is reliable, and that they can leverage it to make more informed and better decisions.

If that’s something you’d like to learn more about, let’s have a conversation.

The post 10 Data Storage Considerations for Growing Companies appeared first on The SERO Group.