The Internet of Things and data storage overload
Organisations of all shapes and sizes are taking interest in the possibilities offered by the Internet of Things (IoT) – and for good reason. According to the McKinsey Global Institute, IoT business will deliver $6.2 trillion worth of revenue by 2025, with over 212 billion ‘things’ connected by 2020. Andri Selamat of Ricoh IT Services explains how you can prepare for the expected data storage challenge ahead.
Millions of devices – from security cameras, activity monitors, home lighting, traffic monitors to smartphones, wearable tech and much more – are permanently connected to the internet. These devices, some of which were previously only connected to power outlets, are now continuously sending and receiving data.
The advent of IoT devices, network traffic and data will exponentially increase the amount of data your organisation needs to receive, store and process. While each device won’t necessarily send much data, Cisco estimates that by 2018 the combined annual amount of data sent will be 403 zettabytes. That’s 403 billion terabytes!
To give you a comparison of the actual scale of 403 zettabyes, consider this: the combined space of the entire World Wide Web was estimated to be close to 500 exabytes in 2009, according to The Guardian – that’s half of one zettabyte.
While the technology exists to collect and analyse IoT data, the storage of this data will fast become an issue if you do not plan carefully for your future data storage needs.
Sophisticated IoT platforms now enable organisations to embrace the variety and volumes of IoT data. Typically quick and low cost to deploy, these technologies facilitate connectivity between data across entire organisations and beyond – irrespective of protocols or APIs. Plus the scalability they offer helps you keep pace with the growth of IoT data.
Let’s take a look at an example of IoT technology at work in the real world – and its implications for data storage systems.
Learn how Ricoh IT Services can help you > See customer stories
A real-world example of IoT data storage
By building a network of internet-connected sensors and devices, planners of ‘smart cities’ are set to provide their populations with better quality of life. The opportunities are endless: enhanced cameras can improve the safety and security of visitors and residents; citizens are able to interact more frequently and meaningfully; resources, utilities and a range of devices from rubbish bins to traffic signals can be better managed.
As you can imagine, with growing populations and increasing number of interconnected devices, storage requirements will increase significantly. Data management and scaling is likely to become costly and time consuming, so these smart cities will need to consider how they can house this influx of data at an affordable cost.
We need a new model for data centres
There’s a growing expectation that the design and nature of data centres must evolve to accommodate the volume of traffic and data from IoT devices. One suggestion is a distributed model of tiered data centres: smaller ones to receive IoT data, then forwarding it over WAN links to a central location.
- Big data centres or ‘motherships’ will become repositories of applications and data
- Distribution data centres will act as regional hubs to move data up from the local level to motherships
- Micro data centres comprising one or two racks will be located close to device population centres so they can receive and transmit data to IoT sensors instantly
The distribution of data centres will have a major impact on how and where data is stored, in what format (RAM, disk, archive), and for how long. Data backup, data restoration and Disaster Recovery pose significant challenges when petabytes of data could potentially take days to restore or recover: how much will it cost you to be without this data?
Utilising the cloud
For many organisations, the cloud presents a potential solution to storage of IoT device data. A cloud storage provider will have the infrastructure to receive data from large numbers of IoT devices and the ability to scale your storage as more data is received.
It’s worth noting that IoT devices may provide data to cloud-based applications, which will make a cloud storage solution even more viable. With IoT data traffic separated from your enterprise network and being transmitted straight to a cloud storage provider, pressure on your own bandwidth limitations can be eliminated.
However, cloud storage will not be appropriate in all cases. When sensitive or personal customer information is being captured, it may be necessary to store IoT data onsite. Evaluation of onsite or cloud IoT storage must be undertaken on a case-by-case basis, with reference to your compliance requirements and corporate information security policy.
Secure data is the foundation of IoT data storage
IoT data is captured in real-time and typically available for analysis immediately. With the potential to capture terabytes of data on a daily basis from a wide variety of IoT devices, a clear and unambiguous approach to data security is critical.
The right technology solution will define and support a consistent approach to ensuring your variety of systems, rights access, devices and networks do not compromise the integrity of the data. Four basic principles of this approach are:
- Authentication
- Authorisation and Role Management
- Message Encryption
- Service Tokens
Once you understand how your IoT data can be stored securely, you can begin to design the best solutions for your specific business needs.
Planning for the next steps in IoT storage
If you’re already collecting data from devices, then you’ll have seen the challenges in managing data growth, storage capacity and the flow and integrity of data.
The cloud offers a realistic option for storing increasing amounts of IoT data, ensuring your corporate network is not unduly flooded with IoT traffic. In order to better understand how cloud-based technologies can be leveraged to satisfy your IoT applications and storage requirements, consider this three-step approach to defining your organisation’s IoT storage needs:
STEP 1 IoT technology and application audit – an enterprise-wide review of current and future IoT applications. Just some of the questions to be asked include what storage technologies are in place, what can be shared and how scalable they are.
STEP 2 IoT data audit – having established how IoT is implemented, what are the current and future data requirements? Storage data growth, security and availability need to be assessed.
STEP 3 IoT gap analysis – given the results of these audits, where are the technology gaps that can be solved internally? Should cloud services be used to hold and present your organisation’s IoT data?
You can then use the results from this three-step process to create an IoT technology roadmap for your organisation, to feel confident you’ll have the storage capability to meet the growing demands of IoT applications.
About the Author
Andri Selamat is Practice Principal at Ricoh IT Services. With almost 20 years’ experience in the ICT sector, he’s currently responsible for the delivery of engineering and architecture services across cloud computing, Microsoft, Citrix and VMware.