Quartz 4

GENAI

2 min read

GENAI.md - Generative AI Usage Declaration

Assignment: DE2 - Lab 2: Text Processing & Inverted Index
Course: Data Engineering II - ESIEE Paris 2025-2026
Track: D - Aviation (METAR weather reports)

1. AI Tool Used

  • Tool: Gemini and Claude
  • Date of usage: April 2026

2. Scope of AI Assistance

The AI was used specifically as a technical assistant for the following tasks:

  1. WSL2 Infrastructure Debugging: Fixing the Spark UI connectivity issues. The AI helped configure spark.driver.host and spark.ui.bindAddress using the socket library to ensure the UI was accessible via localhost:4040 from the Windows browser.
  2. Memory Management: Troubleshooting Java heap space errors. The AI recommended specific memory allocations (spark.driver.memory and spark.executor.memory set to 4GB) to handle the Aviation track’s data volume.
  3. Technical Interpretation: Assistance in identifying and explaining key metrics within the Spark UI, such as the InMemoryTableScan node in the DAG and the memory spill during the aggregation phase.

3. Academic Integrity Statement

We confirm that we understand every line of the code submitted. All critical design decisions regarding the index schema, stop-word filtering, and optimization were made by the student team. The AI served as a debugging and documentation aid for specific challenges.

Declared by Justine Guirauden & Volcy Desmazures — ESIEE Paris, April 2026

Graph View