Advanced Machine Learning Based File Storage Model for Hadoop Dynamic File Access in Bigdata Analytics

Advanced Machine Learning Based File Storage Model for Hadoop Dynamic File Access in Bigdata Analytics
  IJETT-book-cover           
  
© 2023 by IJETT Journal
Volume-71 Issue-6
Year of Publication : 2023
Author : Yallapragada Ravi Raju, D. Haritha, R. Phani Vidyadhar, Mohammed Ayad Alkhafaji, K saikumar, Ahmed J. Obaid
DOI : 10.14445/22315381/IJETT-V71I6P233

How to Cite?

Yallapragada Ravi Raju, D. Haritha, R. Phani Vidyadhar, Mohammed Ayad Alkhafaji, K saikumar, Ahmed J. Obaid, "Advanced Machine Learning Based File Storage Model for Hadoop Dynamic File Access in Bigdata Analytics," International Journal of Engineering Trends and Technology, vol. 71, no. 6, pp. 335-345, 2023. Crossref, https://doi.org/10.14445/22315381/IJETT-V71I6P233

Abstract
Modern technologies manage bigdata data storage applications, which improves application storage. Bigdata surveys simplify file storage. The survey finds no viable file management mechanisms. Existing approaches store unstructured and organized files insecurely. Bigdata analytics requires complex file management. This study implements map secure reduce layers (MSR) and elastic net regression (ENR). MSR-ENR approaches are tested using the HDFS file-handling infrastructure. MSR-ENR can handle all memory file types and extensions (. dox, docs, .pdf, .rar, etc..). Finalize processing time, sensitivity, accuracy, throughput, and recall. This MSR-ENR approach surpasses simulations, challenging existing technology. Big data platforms maintain the cloud, servers, and Hadoop. Data-driven Hadoop modelling cannot provide dynamic actions. App weaknesses include latency and storage. Big data platforms and the Internet have not guided cloud storage upkeep. Big data cloud gateways will drive development and change. This study displays the current method (DL-enabled operational Facilities) through sponsorship software. Intelligent closed-loop video surveillance may speed up and improve large data file maintenance. This speeds up cloud-based large-file production. U-net uses Hadoop and Sparks to analyze data. This software uses Python 3.7. This U-net big data analytics software is competitive.

Keywords
Big data, Hadoop, MSR, ENR, DL-enabled operational Facilities.

References
[1] Hung-Ming Chen, Kai-Chuan Chang, and Tsung-Hsi Lin, “A Cloud-Based System Framework for Performing Online Viewing, Storage, and Analysis on Big Data of Massive BIMs,” Automation in Construction, vol. 71, pp. 34-48, 2016.
[CrossRef] [Google Scholar] [Publisher Link]
[2] Mohd Abdul Ahad, and Ranjit Biswas, “Dynamic Merging based Small File Storage (DM-SFS) Architecture for Efficiently Storing Small Size Files in Hadoop,” Procedia Computer Science, vol. 132, pp. 1626-1635, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[3] Yongqiang He et al., “RCFile: A Fast and Space-Efficient Data Placement Structure in MapReduce-based Warehouse Systems,” IEEE 27th International Conference on Data Engineering, pp. 1199-1208, 2011.
[CrossRef] [Google Scholar] [Publisher Link]
[4] Mohd Rehan Ghazi, and Durgaprasad Gangodkar, “Hadoop, MapReduce and HDFS: A Developers Perspective,” Procedia Computer Science, vol. 48, pp. 45-50, 2015.
[CrossRef] [Google Scholar] [Publisher Link]
[5] K. R. Krish et al., “On Efficient Hierarchical Storage for Big Data Processing,” 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, IEEE, pp. 403-408, 2016.
[CrossRef] [Google Scholar] [Publisher Link]
[6] Herodotos Herodotou et al., “Starfish: A Self-tuning System for Big Data Analytics,” CIDR, vol. 11, no. 2011, pp. 261-272, 2011.
[Google Scholar] [Publisher Link]
[7] Longbin Chen et al., “E2FS: An Elastic Storage System for Cloud Computing,” The Journal of Supercomputing, vol. 74, no. 3, pp. 1045-1060, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[8] Gunasekaran Manogaran et al., “A New Architecture of Internet of Things and Big Data Ecosystem for Secured Smart Healthcare Monitoring and Alerting System,” Future Generation Computer Systems, vol. 82, pp. 375-387, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[9] Jin Wang et al., “Big Data Service Architecture: A Survey,” Journal of Internet Technology, vol. 21, no. 2, pp. 393-405, 2020.
[Google Scholar] [Publisher Link]
[10] Zhihui Lu et al., “IoTDeM: An IoT Big Data-Oriented MapReduce Performance Prediction Extended Model in Multiple Edge Clouds,” Journal of Parallel and Distributed Computing, vol. 118, pp. 316-327, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[11] Jifu Guo, Chunlin Huang, and Jinliang Hou, “A Scalable Computing Resources System for Remote Sensing Big Data Processing Using GeoPySpark Based on Spark on K8s,” Remote Sensing, vol. 14, no. 3, p. 521, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[12] Ahmad Latifian, “How Does Cloud Computing Help Businesses to Manage Big Data Issues,” Kybernetes, vol. 51, no. 6, pp. 1917-1948, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[13] Zhang Xiang, “Visual Dynamic Simulation Model of Unstructured Data in Social Networks,” Security and Communication Networks, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[14] Tianru Zhang, Salman Toor, and Andreas Hellander, “Efficient Hierarchical Storage Management Framework Empowered by Reinforcement Learning,” Arxiv Preprint arXiv:2201.11668, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[15] Maarten Jansen, “On-Board-Unit Big Data Analytics: from Data Architecture to Traffic Forecasting,” Doctoral Dissertation, Katholieke Universiteit Leuven, 2022.
[Google Scholar] [Publisher Link]
[16] Vijayalakshmi Saravanan, Fatima Hussain, and Naik Kshirasagar, “Role of Big Data in Internet of Things Networks,” Research Anthology on Big Data Analytics, Architectures, and Applications, IGI Global, pp. 336-363, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[17] Sundeep Kumar, Vikram Singh Rathore, and Alok Mathur, “An Analytical Study on Big Data Management for Supply Chain Analytics,” Recent Advances in Industrial Production, Springer, Singapore, pp. 333-341, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[18] Roberto Cavicchioli, Riccardo Martoglia, and Micaela Verucchi, “A Novel Real-Time Edge-Cloud Big Data Management and Analytics Framework for Smart Cities,” Journal of Universal Computer Science, pp. 3-26, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[19] P. Manivannan, D. Prabha, and K. Balasubramanian, “Artificial Intelligence Databases: Turn-on Big Data of the SMBs,” International Journal of Business Information Systems, vol. 39, no. 1, pp. 1-16, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[20] Loris Belcastro et al., “Programming Big Data Analysis: Principles and Solutions,” Journal of Big Data, vol. 9, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[21] Abolgasem M. Ali Enfais et al., “Enhancing Hadoop Performance in Homogeneous Big Data Environment Assuming Configuration of Dynamic Slots in Map-Reduce Pattern,” International Journal of Engineering & Technology, vol. 7, no. 4, pp. 6986-6990, 2018.
[Google Scholar] [Publisher Link]
[22] Sudhir Allam, “An Exploratory Survey of Hadoop Log Analysis Tools," International Journal of Creative Research Thoughts, vol. 6, no. 8, pp. 801-804, 2018.
[Google Scholar] [Publisher Link]
[23] Laouni Djafri, “Dynamic Distributed and Parallel Machine Learning Algorithms for Big Data Mining Processing,” Data Technologies and Applications, vol. 56, no. 4, pp. 558-601, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[24] H K Pradeep, K Rohitaksha, and C B Abhilash, "An Email based Offline Download Manager for Large Distributed File System using Hadoop MapReduce Framework," SSRG International Journal of Computer Science and Engineering, vol. 1, no. 10, pp. 1-5, 2014.
[CrossRef] [Google Scholar] [Publisher Link]
[25] Ines Sbai, and Saoussen Krichen, “A Real-Time Decision Support System for Big Data Analytic: A Case of Dynamic Vehicle Routing Problems,” Procedia Computer Science, vol. 176, pp. 938-947, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[26] Bunmi Deborah Millennial-Oriagbo, and Muhammad Ghali Aliyu, "Pros and Cons of Big data in a Global Digital Transformation," SSRG International Journal of Mobile Computing and Application, vol. 8, no. 3, pp. 1-10, 2021.
[CrossRef] [Publisher Link]
[27] O Sai Saran et al., “3D Printing of Composite Materials: A Short Review,” Materials Today: Proceedings, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[28] Kiran Dasari, Lokam Anjaneyulu, and Jayaraju Nadimikeri, “Application of C-Band Sentinel-1A SAR Data as Proxies for Detecting Oil Spills of Chennai, East Coast of India,” Marine Pollution Bulletin, vol. 174, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[29] K. Sarada et al., “Records of Patient Health Data and Medical Information Monitoring Using IOT,” 2nd International Conference for Innovation in Technology, IEEE, pp. 1-6, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[30] S. Murugan et al., “Impact of Internet of Health Things (IoHT) on COVID-19 Disease Detection and Its Treatment Using Single Hidden Layer Feed Forward Neural Networks (SIFN),” How COVID-19 is Accelerating the Digital Revolution: Challenges and Opportunities, Cham: Springer International Publishing, pp. 31-50, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[31] Chandra Shaker Pittala, Vallabhuni Vijay, and B. Naresh Kumar Reddy, “1-Bit FinFET Carry Cells for Low Voltage High-Speed Digital Signal Processing Applications,” Silicon, vol. 15, pp. 713-724, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[32] Uluer Emre Özdil, and Serkan Ayvaz, “An Experimental and Comparative Benchmark Study Examining Resource Utilization in Managed Hadoop Context,” Cluster Computing, vol. 26, pp. 1891-1915, 2021.
[CrossRef] [Google Scholar] [Publisher Link]