Data Proliferation

Data Proliferation

Data Proliferation Jonathan Poland

Data proliferation refers to the rapid growth of data, often resulting in a large amount of replicated and low-quality data. This can be costly to manage and may pose compliance and operational risks to an organization. While it may be necessary to analyze this data in order to understand its structure, sources, and uses, it may ultimately have little value to the organization and can be difficult to discard. The following are illustrative examples of data proliferation.

Customer Data

It is common for multiple systems in an organization to maintain customer data. Such data is commonly out of sync between systems with no clear single source of truth. This can cause operational failures such as sending a bill to the wrong address.

Documents

Knowledge workers tend to create a lot of documents that get checked into a document management system. In many cases, such documents become completely unused with time but are retained as a precaution.

Communication

Communications such as emails can gather at the rate of hundreds per employee per day. Most communications lose their value almost immediately but often are retained for an extended period of time.

Backups

Backups of data, documents and communications often need to be retained in case something important was deleted from the source systems. If someone deletes a critical email, the only copy may be in a backup from a particular day last year. As such, backups are commonly stored for long periods of time. This can consume considerable resources despite the fact that backups are rarely used.

Transactional Data

Transactional data such as market trades and website purchases can grow extremely quickly. Transactional data is often viewed as valuable for historical research. For example, it is common to look at patterns in stock trades going back decades.

Social Data

Data that is shared by people on a public or private social network. Often viewed as valuable for purposes such as market research and machine learning.

Sensors & Machines

Machine and sensor generated data. Sensors have become cheap to the extent than they can be embedded in everyday objects in great numbers. Such data may be generally less valuable than human generated data. For example, video of a train tunnel or data from a tire pressure sensor isn’t interesting for long. Nevertheless, sensor data potentially represents a gigantic source of data that is far larger than all other sources combined.

Learn More
Continuous Improvement Jonathan Poland

Continuous Improvement

Continuous improvement is a systematic approach to improving products, services, and processes over time. It involves a cycle of planning,…

Sales Objections Jonathan Poland

Sales Objections

A sales objection is a concern or hesitation that a customer has about making a purchase. Identifying and addressing these…

Microtransactions Jonathan Poland

Microtransactions

Microtransactions is a large scale industry that is becoming a dominant business for certain types of companies. They are small…

Systems Thinking Jonathan Poland

Systems Thinking

Systems thinking is the practice of analyzing the entire system, rather than just its individual parts, in order to understand…

Mission Statement Jonathan Poland

Mission Statement

A mission statement is a statement of purpose that defines the goals and values of an organization. It is a…

Sales Management Jonathan Poland

Sales Management

Sales management is the process of overseeing and directing an organization’s sales team. It involves setting sales goals, analyzing data,…

Business Decisions Jonathan Poland

Business Decisions

A business decision is a commitment made by a company, team, or individual employee to a specific course of action.…

Lifecycle Cost Analysis Jonathan Poland

Lifecycle Cost Analysis

Lifecycle cost analysis is a tool used to evaluate the total cost of owning and operating a product, system, or…

Good Failure Jonathan Poland

Good Failure

Good failure, also known as productive failure, refers to the idea that failure can be a valuable learning experience and…

Content Database

Economic Security Jonathan Poland

Economic Security

Economic security refers to the ability of an individual or a household to meet their basic needs, such as food,…

Market Research 150 150 Jonathan Poland

Market Research

Market research is a fundamental step for business development as it helps businesses understand their market, customers, and competitors better.…

A/B Testing Jonathan Poland

A/B Testing

A/B testing, also known as split testing or experimentation, is a statistical method used to compare two versions of a…

Life Skills Jonathan Poland

Life Skills

Life skills are essential abilities that enable individuals to navigate the complexities of daily life and achieve their goals. These…

Quality Assurance Jonathan Poland

Quality Assurance

Quality assurance (QA) is the process of verifying that a product or service meets specific quality standards. This is often…

Team Strategy Jonathan Poland

Team Strategy

A team strategy is a plan that outlines how a team will achieve its goals. Developing and implementing a strategy…

Technology 101 Jonathan Poland

Technology 101

Technology is an important component of every business, constantly reshaping entire industries. Keeping pace with new and emerging technology can…

Payback Theory Jonathan Poland

Payback Theory

Let’s say you live in a town with two bakeries for sale at $1 million each. Both offer similar products…

What are Finished Goods? Jonathan Poland

What are Finished Goods?

Finished goods are products that have completed the manufacturing process and are ready for sale to customers. They are the…