Building a big data platform using software without licence costs

Vassilev, Vassil, Sowinski-Mydlarz, Viktor, Gasiorowski, Pawel, Radu, Sorin, Nakarmi, Sabin, Hristev, Martin, Baghaeishiva, Reza and Bali, Tarun (2023) Building a big data platform using software without licence costs. In: Open-source horizons - Challenges and opportunities for collaboration and innovation. IntechOpen, London, pp. 1-24.

[img]
Preview
Text
Final CHAPTER VERSION.pdf - Published Version
Available under License Creative Commons Attribution.

Download (7MB) | Preview
Official URL: https://www.intechopen.com/online-first/1155033

Abstract / Description

This chapter presents the experience in developing and utilizing Big Data platforms using software without license costs, acquired while working on several projects at two research institutions – the Cyber Security Research Centre of London Metropolitan University in the United Kingdom and the GATE Institute of Sofia University in Bulgaria. Unlike the universal computational infrastructures available from large cloud service providers such as Amazon, Google, Microsoft and others, which provide only a wide range of universal tools, we implemented a more specialized solution for Big Data processing on a private cloud, tailored to the needs of academic institutions, public organizations and smaller enterprises which cannot afford high running costs, or do significant in-house development. Since most of the currently available commercial platforms for Big Data are based on open-source software, such a solution is fully compatible with enterprise solutions from leading vendors like Cloudera, HP, IBM, Oracle and others. Although such an approach may be considered less reliable due to the limited support, it also has many advantages, making it attractive for small institutions with limited budgets, research institutions working on innovative solutions and software houses developing new platforms and applications. It can be implemented entirely on the premises, avoiding cloud service costs and can be tailored to meet the specific needs of the organizations. At the same time, it retains the opportunity for scaling up and migrating the developed solutions as the situations evolve.

Item Type: Book Section
Uncontrolled Keywords: big data; AI; data platform; private cloud; public domain
Subjects: 000 Computer science, information & general works
Department: School of Computing and Digital Media
Depositing User: Vassil Vassilev
Date Deposited: 04 Dec 2023 09:50
Last Modified: 04 Dec 2023 09:50
URI: https://repository.londonmet.ac.uk/id/eprint/8935

Downloads

Downloads per month over past year



Downloads each year

Actions (login required)

View Item View Item