Yousry Mohamed – Medium

Yousry Mohamed

Published in
Level Up Coding

Stop using plain PySpark UDFs : No one likes slow cars! Part II

Quick insights about how Pandas UDFs work

Sep 7

Stop using plain PySpark UDFs : No one likes slow cars! Part II

Sep 7

Published in
Level Up Coding

Stop using plain PySpark UDFs : No one likes slow cars!

How complex logic can be still implemented using out of the box Spark functions with lightning fast performance.

Jul 18

Stop using plain PySpark UDFs : No one likes slow cars!

Jul 18

Published in
Level Up Coding

Delta Lake Liquid Clustering — A visual explanation

How to optimize lakehouse data storage layout with minimal effort.

Jan 28

Delta Lake Liquid Clustering — A visual explanation

Jan 28

Published in
Level Up Coding

Building a lakehouse on Google Cloud sans Databricks

Combining Delta Lake, Iceberg and BigLake

Dec 17, 2023

Building a lakehouse on Google Cloud sans Databricks

Dec 17, 2023

Published in
Level Up Coding

Navigating the Void: Unraveling the Mysteries and Pitfalls of the ‘Void’ Data Type in Apache Spark

Exploring how a void column appears into Spark DataFrames and what could be the implications.

Dec 5, 2023

Navigating the Void: Unraveling the Mysteries and Pitfalls of the ‘Void’ Data Type in Apache Spark

Dec 5, 2023

Published in
Level Up Coding

Setting up a PySpark local developmet environment for Dataproc serverless

Smooth dependency managment for local developmet and production jobs

Aug 29, 2023

Setting up a PySpark local developmet environment for Dataproc serverless

Aug 29, 2023

Published in
Level Up Coding

dbt tests vs Delta Live Tables expectations : a click bait to Spark observable metrics

Comparing dbt and DLT tests performance and corelating to Spark observable metrics

Aug 5, 2023

dbt tests vs Delta Live Tables expectations : a click bait to Spark observable metrics

Aug 5, 2023

Published in
Level Up Coding

Delta Lake Universal Format — A First Look

Write as delta lake — Read as iceberg

Jul 15, 2023

Delta Lake Universal Format — A First Look

Jul 15, 2023

Published in
Level Up Coding

Back to basics : Spark caching key ideas!

Foundational concepts about how Spark caches Dataframes vs RDDs

Mar 13, 2023

Back to basics : Spark caching key ideas!

Mar 13, 2023

Delta lake Z-Ordering from A to Z

Understand how to optimise delta lake tables for high cardinality queries.

Sep 19, 2022

Delta lake Z-Ordering from A to Z

Sep 19, 2022

Yousry Mohamed

Yousry Mohamed

Yousry is a lead consultant working for Cuusoo. He is very passionate about all things data including Big Data, Machine Learning and AI.

Following

Help
Status
About
Careers
Press
Blog
Privacy
Terms
Text to speech
Teams