Data Collection: Two Key Tools to Improve Your Data Strategy

Are your company’s data collection processes sound? Do they align with best practices?

Welcome to the first of three posts on how to refine your strategy for data lifecycle management. In this post, we will look at how to evaluate your data collection processes for improvements.

Data Collection in Data LifeCycle Management (DLM)

As has been noted in a previous post on the difference between Data Lifecycle Management (DLM) and Information Lifecycle Management(ILM), there are fundamentally 3 phases of Data Lifecycle Management (DLM) into which all physical data-related tasks fall: Data Collection & Creation, Data Management, and Data Deletion.

As an IT leader, there are two important exercises you should perform to evaluate your data collection strategy. By performing these, you will produce living documents that should guide how your company creates, ingests, and consumes data now and in the future.

First, perform a data collection audit.

The first step in evaluating your DLM processes is to gain a complete understanding of the data collection processes your company is currently using. The best way to do this is through an internal audit.

Your data collection audit should include answers to the following questions:

• Where is data coming into your systems (websites, transactional systems, vendors)?
• What systems or processes are used to create or collect data (software, web forms, APIs, FTP)?
• What data formats are being leveraged by these processes (SQL, JSON, CSV, XML)?
• What security, threat mitigation, backups, and archiving processes are in place for these processes and data stores?  (Here is a good summary of what to look for.)

Finally, examine the information you gathered through a strategic lens. Look for vulnerabilities, inefficiencies, and pain points in your processes. Then work with your team to devise a strategic plan and implementation timeline for achieving improvements in these areas.

Second, create a data tracking plan.

Now that you have audited your data collection processes, you should give thought to why you are collecting the data that you are and what data needs to be collected. Consult business analysts in your company about which metrics they would like to track. This will help you understand what data points need to be collected. Likewise, find out what government regulations dictate about what data should be collected and retained.

Ask questions like:

• Is the right data being collected?
• Are there any missing data points?
• Are you collecting irrelevant or duplicated data?

Bridge the Gap

Undoubtedly, it is tricky to bridge the gap between the business, which has ideas about what data they would like to track, and the technical team, who knows how to track it. A data tracking plan is a tool that can help with this.

While some people strictly define what a data tracking plan must consist of, a simple plan is often sufficient. Your data tracking plan defines your primary business objects (customers, products, stores, etc.) and the metrics or events surrounding them that your business would like to have more information about.

Before spending time creating your own, take a look at the many templates available to get you started. Here is a link to an evaluation of a few free templates to start your research.

Create the plan

Once you have your template, start your internal planning discussions with questions like:

• What core business objects are we concerned with?
• What metrics do we care about for those objects (that is, what do we want to track about them)?
• Why do we want to track these metrics?
• What data needs to be collected to obtain these metrics and how will it be defined?
• Where can the data be obtained? Do we already collect it?
• Who will govern the information once we have it?
• Who will manage the data collection?
• What format does the data need to be in to be useful?

It can certainly be challenging on many fronts for the business and IT to come together to create a data tracking plan. However, facilitating this will be well worth the effort in terms of the clear data strategy objectives that will be produced. Avoiding the costs associated with misguided data projects will more than outweigh the time and energy spent in coordinated planning.

Finally, update your data strategy, and implement changes.

Once you have assessed your data collection processes and have identified improvements, you’ll need to assign priorities to your findings. Work with both the business and your technical team to set these priorities, as well as to build a roadmap for implementation.

As can be seen, the information you have gathered through cross-functional cooperation and through using these tools will help you to make a strong case to business leaders for the importance of these strategic improvements.

Want to learn more?

Looking for more information about Data Strategy and how it can help align IT and business goals? Check out these posts.

If you’d like to learn more about how we approach Data Strategy, or if you have some concerns about your SQL estate, give us a call. We can help.