In the age of big data and advanced analytics, many organisations have been on a trend of hoarding as much data as they possibly can. Many have millions of spreadsheets, files, documents and emails tucked away in various locations, but at a certain point in time this data loses its business value. It becomes what is known as dark data.
Dark data is data that has outlived its usefulness and no longer provides any value for your business. Continuing to store this data can become very risky, and potentially very costly. One recent study found that, on average, 54% of an organisation’s data is considered dark data, which leads to additional costs, security risks and wasted resources.
This dark data can take many forms, including outdated:
- Log Files
- Account Information
- Employee Information
- Customer Records
- Financial Data
- Email Records
- Notes and Presentations
- Earlier Iterations of Current Documents
How Can Dark Data Be Harmful?
Dark data has many issues with it. The first of which is that it threatens your storage resources and as more dark data is collected, there is less room for the current, relevant data – unless you want to purchase more storage, which leads to a greater overhead cost. Further, many organisations will identify dark data as a lower-security priority and when this happens, attackers may gain access to some valuable information, even if you do consider it outdated for your business operations.
Compliance can also become a huge issue when you have excessive dark data. The more you have, the harder it becomes to protect, and if your dark data files include employee information or customer payment information, you may be violating regulations in addition to putting older employees and customers at risk. Storing dark data, or failing to adequately protect it, opens the door to security breaches which often result in financial losses and a loss in consumer confidence.
How Can Dark Data be Managed?
The importance of protecting dark data and reducing the amount of dark data that companies store will continue to grow as the ability to amass more and more data increases. So, even if you aren’t thinking about how your dark data can impact your organisation now, you will need to address it sometime – and likely in the near future.
There is no way to eliminate dark data entirely as all data generally goes through a transition period in which its usefulness is questioned, but not necessarily decided on yet. Yet, there are ways to reduce the amount dark data currently held at your organisation and monitor it so that the threat of a security or compliance risk is minimised. These include:
- Take stock of all of your data. This means auditing your entire data ecosystem and removing or archiving old, unnecessary data. This process should be done periodically to prevent dark data from accumulating excessively.
- Prune the data, don’t dump it. Rather than doing a total data dump of all old material, it is worth going through it to find out what is worth keeping and what is not. When you find something worthwhile, put it in a manageable format so that it is easy to access when it is needed.
- Prioritise and limit access. When older data does need to be retained, it is worth restricting access. This will reduce the likelihood of a security breach stemming from older files that contain sensitive information.
- Assess how you backup your data. Organisations always have backup data. Frequently, this means making a full backup of your database – which increases your dark data problems since you are duplicating all of the useless information repeatedly. A better backup solution takes one replica of the database and then makes incremental backups of active data locations over time.
- Encrypt your data. Data encryption is a part of any data management strategy, but it isn’t always extended to dark data. When it is, it can go a long way in ensuring this dark data doesn’t ultimately hurt your organisation.
- Assess your resources on the cloud. It’s easy to assume all data is safe on the cloud, but the reality is that the security precautions taken here must be monitored just as any other data storage would be.
Dark data will pose a growing threat to organisations in the age where there is an emphasis on amassing more and more data.
Modern intelligent content management solutions are incredibly effective tools in the battle against dark data. They allow for the setting of expiratory dates on documents and data points as well as automated workflows. This helps to ensure that the data stored is the data you need – and that you aren’t keeping data that can put you at unnecessary risk.
