Working with edge devices and high-end applications can have several unpredictable results and problems. For example, if you have an application that needs to download and deploy 50GB images from a repository in semi-real time you cannot rely on the network bandwidth or the internet connection quality of the edge device, chances are that the connection would prove unstable or just inadequate.
That is one of the reasons we have to move the data storage to the edge. Edge devices could exchange data within their local network, releasing internet bandwidth for other purposes. Moreover, the local area networks are steadier, involving fewer nodes than trying to reach a remote repository. Finally, edge storage can and should be decentralized in nature, providing data security, fault tolerance, disaster recovery and peer-to-peer data transfer capabilities.
In ACCORDION data storage is moved to the edge, forming private repositories within each edge device cloud (a.k.a. mini-cloud ). The edge storage platform developed is designated as the ACCORDION Edge Storage component ( ACES ). ACES is based on the MinIO platform, using a lightweight Kubernetes ( K3s ) version as an orchestrator. It creates a decentralized distributed block storage service that is placed on one or more of the edge devices that form the mini-cloud. This storage is accessible by all devices of the mini-cloud through several ways including a web-based interface, a CLI tool and an S3 API.
Moreover, new pods being deployed in the mini-cloud can access ACES through one or more persistent volume claims that are created and connected to the MinIO servers. The connection and configuration are performed using deployment scripts based on the Datashim.io platform tools. Datashim provides a collection of data serving tools, building on the Kubernetes Container Storage Interface (CSI), which enables the attachment of remote storage options as local disk devices. This enables pods to seamlessly mount remote storage repositories as filesystem paths and access them through their filesystem.
The evaluation phase of ACES is expected to start in the immediate future, examining a wide range of key point indicators. The evaluation will examine the data transfer speed, integrity and availability for data demanding applications. We will also examine the platform’s fault tolerance and disaster mitigation mechanisms by creating artificial faulty scenarios like an edge device overheating, disconnecting or becoming unreliable for some reason. The scenarios that will be tested include the transfer of a few big files ( ~ 50GB ) as well as the transfer of many small files ( ~ 500kb). The first scenario tests the performance and general behaviour of the system for applications that are bundled in big images while the second tests the behaviour for application that are un-bundled, requesting many smaller files like image sprites, sound files, source code files e.t.c.
Author: John Violos | ICCS