Build scalable and reliable data ecosystems using Data Mesh Databricks Spark and KafkaKey FeaturesDevelop modern data skills used in emerging technologiesLearn pragmatic design methodologies such as Data Mesh and data lakehousesGain a deeper understanding of data governancePurchase of the print or Kindle book includes a free PDF eBookBook DescriptionModern Data Architectures with Python will teach you how to seamlessly incorporate your machine learning and data science work streams into your open data platforms You ll learn how to take your data and create open lakehouses that work with any technology using tried and true techniques including the medallion architecture and Delta Lake Starting with the fundamentals this book will help you build pipelines on Databricks an open data platform using SQL and Python You ll gain an understanding of notebooks and applications written in Python using standard software engineering tools such as git pre commit Jenkins and Github Next you ll delve into streaming and batch based data processing using Apache Spark and Confluent Kafka As you advance you ll learn how to deploy your resources using infrastructure as code and how to automate your workflows and code development Since any data platform s ability to handle and work with AI and ML is a vital component you ll also explore the basics of ML and how to work with modern MLOps tooling Finally you ll get hands on experience with Apache Spark one of the key data technologies in today s market By the end of this book you ll have amassed a wealth of practical and theoretical knowledge to build manage orchestrate and architect your data ecosystems What you will learnUnderstand data patterns including delta architectureDiscover how to increase performance with Spark internalsFind out how to design critical data diagramsExplore MLOps with tools such as AutoML and MLflowGet to grips with building data products in a data meshDiscover data governance and build confidence in your dataIntroduce data visualizations and dashboards into your data practiceWho this book is forThis book is for developers analytics engineers and managers looking to further develop a data ecosystem within their organization While they re not prerequisites basic knowledge of Python and prior experience with data will help you to read and follow along with the examples