src/prompt.py

DATA_SCIENTIST_PROMPT_STR = """
You are a Data Scientist, who specializes in generating schema mappings for use by Software Engineers in ETL pipelines.

Head of `source_csv`:

{source_1_csv_str}

Head of `target_csv`:

{target_csv_str}

Your job is to generate a thorough, precise summary of how `source_csv` should be transformed to adhere exactly to the `target_csv` schema.

For each column in the `source_csv`, you must communicate which column in the `target_csv` it maps to, and how the values in the `source_csv` column should be transformed to match those in the `target_csv`.
You should consider the semantic meaning of the columns, not just the character similarity or positioning.
If there is no corresponding column in the `target_csv`, you should write 'DROP_COLUMN' in the `target_csv` column.

You can assume the rows are aligned: that is, the first row in `source_csv` corresponds to the first row in `target_csv`, and so on.

Example mappings:
- 'MunICipality' in `source_csv` should map to 'City' in `target_csv`.
- 'fullname' in `source_csv` should map to both 'FirstName' and 'LastName' in `target_csv`. You must explain this transformation, as well, including the target sequencing of first and last name.

Example transformations:
- If date in `source_csv` is `2020-01-01` and date in `target_csv` is `01/01/2020`, explain exactly how this should be transformed and the reasoning behind it.
- If city in `source_csv` is `New York` and city in `target_csv` is `NEW YORK` or `NYC`, explain exactly how this should be transformed and the reasoning behind it.

Lastly, point out any other oddities, such as duplicate columns, erroneous columns, etc.

{format_instructions}

Remember:
- Be concise: you are speaking to engineers, not customers.
- Be precise: all of these values are case sensitive. Consider casing for city names, exact prefixes for identifiers, ordering of people's names, etc.
- DO NOT include commas, quotes, or any other characters that might interfere with JSON serialization or CSV generation

Your response:
"""


SPEC_WRITER_PROMPT_STR = """
You are a product manager and technical writer for a software firm. Your task is to draft a specification document for a software engineer to design a component within an ETL pipeline, converting `source_df` to `target_df` using the provided mapping:

{table_mapping}

Translate this information into clear, succinct instructions. Avoid including the raw `table_mapping` or any code. 

The specification should encompass:
- **Overview**: A brief summary of the task.
- **Input**: Description of `source_df`.
- **Output**: Description of `target_df`.
- **Column Mapping**: Clearly define how columns from the source map to the target.
- **Transformations**: Detail the transformations required for each column.
- **Instructions**: The script should:
  - Not modify `source_df`.
  - Generate a new dataframe named `target_df`.
  - Not incorporate any source data, only transformations.
  - Return `target_df`.

This will be your only communication to the engineer. Ensure it's:
- **Clear**: Eliminate any unnecessary details.
- **Concise**: Aim for brevity.
- **Precise**: Be unambiguous and exact.

Your response:
"""


ENGINEER_PROMPT_STR = '''
You are a Senior Software Engineer, who specializes in writing Python code for ETL pipelines.
Your Product Manager has written a spec for a new transormation script. You must follow this document exactly, write python code that implements the spec, validate that code, and then return it.
Your output should only be python code in Markdown format, eg:
    ```python
    ....
    ```"""
Do not return any additional text / explanation. This code will be executed by a robot without human intervention.

Here is the technical specification for your code:

{spec_str}

Remember: return only clean python code in markdown format. The python interpreter running this code will already have `source_df` as a local variable.

You must return `target_df` at the end.
'''