Open Access Open Access  Restricted Access Subscription Access

From Chaos to Clarity: Building a Python Data ETL Pipeline

K. Sai Rohith, S. Vamshi, M. Bharathi, T. Aditya Sai Srinivas

Abstract


This paper explores the significance of Data ETL pipeline development as a highly valuable skill for Data Engineers. Data ETL encompasses the extraction, transformation, and loading of data from a source into a database. By focusing on the development of a Data ETL pipeline using Python, this article provides a comprehensive guide for those seeking to acquire expertise in this area.


Full Text:

PDF

References


Hou Su, Voon, Sourav Sen Gupta, and Arijit Khan. "Automating ETL and Mining of Ethereum Blockchain Network." In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, pp. 1581-1584. 2022.

Suleykin, Alexander, and Peter Panfilov. "Metadata-driven industrial-grade etl system." In 2020 IEEE International Conference on Big Data (Big Data), pp. 2433-2442. IEEE, 2020.

Biswas, Neepa, Anamitra Sarkar, and Kartick Chandra Mondal. "Efficient incremental loading in ETL processing for real-time data integration." Innovations in Systems and Software Engineering 16 (2020): 53-61.

https://www.analyticsvidhya.com/blog/2022/06/a-complete-guide-on-building-an-etl-pipeline-for-beginners/


Refbacks

  • There are currently no refbacks.