I used AI to help me:
-
Identify and fix Python errors in my shuffle and hash join implementations.
-
Understand the differences between shuffle joins and replicated hash joins, and decide when each is preferable based on input sizes and key distributions.
-
Validate outputs by comparing results from both join methods and ensuring they matched expectations.
-
Debugging my Python code, especially fixing errors with tuples and strings in RDDs.