Skip to main content
The ETL pipeline enables full data migration between ECC and S/4HANA systems. It provides 4 tools that the agent orchestrates in a strict 6-step workflow, with mandatory safety checks at every stage.

​
Overview

The pipeline layers on top of existing SAP primitives (read_table, execute_rfc, get_table_schema). No additional Java connector endpoints are needed — all operations use standard RFC calls. 
Source SAP ──extract──> Transform Engine ──load──> Target SAP (via BAPI)
   (ECC)     read_table    12 converters    execute_rfc    (S/4HANA)
                                               |
                                    BAPI_TRANSACTION_COMMIT
                                    BAPI_TRANSACTION_ROLLBACK

​
Agent workflow

When a user asks the agent to execute a migration, the system prompt enforces this sequence. The agent cannot skip steps 3, 4, or 6 — even if the user asks it to.
1

Field Mapping

Agent calls generate_field_mapping to compare source and target schemas. Returns coverage stats, type conflicts, and unmapped fields.
2

Transform Suggestions

Agent calls suggest_transforms which fetches live schemas from both systems and compares field types/lengths. Returns deterministic suggestions based on schema diffs (NUMC padding, date conversion, currency shifts, etc.). The agent reviews suggestions and asks the user about ambiguous cases — currency shift direction, UOM mapping tables, target client number.
3

Pre-Validation (mandatory)

Agent calls validate_migration. Six checks run against the live systems: source/target readability, target field existence, key field coverage, source data availability, transform type validity. If any error-severity check fails, the agent stops.
4

Dry Run Preview (mandatory)

Agent calls extract_transform_load with dry_run=true. Extracts and transforms data but writes nothing. Returns 5 sample rows showing source vs transformed values. The agent presents these to the user and waits for confirmation.
5

Execute ETL

After user confirms the preview, agent calls extract_transform_load with dry_run=false. Data is loaded in batches via BAPI calls with per-batch commit/rollback.
6

Post-Reconciliation (mandatory)

Agent calls reconcile_migration to compare row counts and spot-check sample keys between source and target. Reports match/mismatch/partial status.

​
Tools

​
suggest_transforms

Fetches schemas from both live SAP systems, compares field types and lengths for each mapped field, and suggests typed transformation rules. Parameters:
ParamTypeDescription
source_tablestringSource SAP table name
target_tablestringTarget SAP table name
source_system's4' | 'ecc'Source system identifier
target_system's4' | 'ecc'Target system identifier
field_mappingarrayField mapping from generate_field_mapping output
How it works: The suggestion logic is deterministic — it walks a priority chain of rules based on the schema diff:
  1. Target field is MANDT or CLIENT → mandt_set
  2. Target type is NUMC and lengths differ → numc_pad (left zero-padding)
  3. One side is DATS, other isn’t → dats_format
  4. One side is TIMS, other isn’t → tims_format
  5. Both CURR but lengths differ → currency_shift
  6. One side is UNIT and types differ → uom_map
  7. Both CHAR, source longer → truncate
  8. Both CHAR, target longer → char_pad
  9. Any other type mismatch → custom_expression (flagged for manual review)
The agent then reviews these suggestions using its SAP domain knowledge — it may override char_pad with numc_pad for known numeric fields, add transforms the tool missed, or flag issues the heuristics don’t catch. Approval required: No

​
validate_migration

Runs 6 pre-flight checks against the live SAP systems before any data moves. Parameters:
ParamTypeDescription
source_tablestringSource SAP table name
target_tablestringTarget SAP table name
source_system's4' | 'ecc'Source system identifier
target_system's4' | 'ecc'Target system identifier
field_mappingarrayField mapping with optional transform rules per field
Checks:
CheckSeverityWhat it does
Source table readableErrorCalls get_table_schema on source system
Target table readableErrorCalls get_table_schema on target system
Target fields existErrorVerifies every mapped target field exists in the live target schema
Key fields coveredErrorEnsures all key fields in the target table (except MANDT) are included in the mapping
Source has dataWarningReads 1 row from source to check the table isn’t empty
Transform types validErrorValidates every specified transform type against the 12 known types
Returns a blocking_failures count. If > 0, the agent refuses to proceed. Approval required: No

​
extract_transform_load

The core ETL tool. Reads from source, transforms, writes to target via BAPI. This is the only ETL tool that requires user approval before execution. Parameters:
ParamTypeDefaultDescription
source_tablestringSource SAP table name
target_tablestringTarget SAP table name
source_system's4' | 'ecc'Source system identifier
target_system's4' | 'ecc'Target system identifier
field_mappingarrayField mapping with optional transform rules
target_bapistringBAPI function module for loading (e.g., BAPI_MATERIAL_SAVEDATA)
bapi_import_templateobject{}Static import parameters for the BAPI
bapi_table_paramstringBAPI table parameter that receives row data
batch_sizenumber200Rows per batch
source_filtersstring[]WHERE clauses for source extraction
dry_runbooleanfalsePreview only — no writes
Execution flow:
Paginated read_table calls against the source SAP system. Reads batch_size rows at a time until fewer rows return than requested, or the 10,000 row safety cap is hit. Source filters pass through as WHERE clauses.
Every source row is transformed field-by-field using the mapping. Fields with a transform rule run through the converter function (pure string-to-string). Fields without a transform copy the value to the target field name.
If dry_run=true, returns 5 sample source and transformed rows and stops. No data is written.
For each batch:
  1. Build BAPI call — transformed rows go into tables_params[bapi_table_param]
  2. Call execute_rfc on the target system’s connector
  3. Check RETURN table for TYPE: 'E' (Error) or 'A' (Abort)
  4. If clean → BAPI_TRANSACTION_COMMIT with WAIT: 'X'
  5. If errors → BAPI_TRANSACTION_ROLLBACK
  6. If exception → catch, attempt rollback, record error
Each batch is an independent transaction. Partial migration (batches 1-3 committed, batch 4 failed) is expected behavior for large datasets.
Returns: Per-batch results (rows attempted/succeeded/failed, errors, committed flag) and overall status: completed, partial, or failed. Approval required: Yes

​
reconcile_migration

Post-migration verification. Reads from both systems and compares. Parameters:
ParamTypeDefaultDescription
source_tablestringSource SAP table name
target_tablestringTarget SAP table name
source_system's4' | 'ecc'Source system identifier
target_system's4' | 'ecc'Target system identifier
key_fieldsstring[]Key fields to compare
sample_sizenumber10Number of keys to spot-check
How it works:
  1. Reads key fields from both source and target (parallel calls, up to 10,000 rows each)
  2. Compares row counts
  3. Samples N evenly-spaced keys from source, builds composite key strings, checks each against a Set of target keys
  4. Reports status:
    • match — counts equal AND all spot-checked keys found
    • mismatch — none of the spot-checked keys found
    • partial — some found, some not
Approval required: No

​
Transform engine

The transform engine (src/tools/etl/transformers.ts) is a set of pure functions with no side effects and no connector calls. Each converter takes a string value and a config object, returns a string.
TransformWhat it doesConfig
numc_padStrip non-digits, left-pad with zeros{ target_length: 40 }
dats_formatConvert between SAP YYYYMMDD and ISO dates{ source_format, target_format }
tims_formatConvert between SAP HHMMSS and HH:MM:SS{ source_format, target_format }
currency_shiftShift decimal places left or right{ shift: 2, direction: 'left' }
uom_mapMap unit of measure codes via lookup table{ mapping: { 'EA': 'ST' } }
truncateCut string to max length{ max_length: 40 }
char_padRight-pad with spaces{ target_length: 20 }
uppercaseForce uppercase{}
default_valueFill empty values{ value: '0000' }
lookupStatic value map{ table: { 'A': '01' } }
mandt_setOverride with target client number{ target_mandt: '100' }
custom_expressionPass-through (safety placeholder){}

​
Agent safety behaviors

The system prompt enforces several safety rules that the agent follows regardless of user input:
  • Ambiguity detection — If the user doesn’t explicitly state source and target systems, the agent asks for clarification before proceeding.
  • Mandatory validation — The agent always runs validate_migration before any ETL execution, including dry runs. It will not proceed if any error-severity check fails.
  • Mandatory dry run — The agent always runs extract_transform_load with dry_run=true first and presents the preview. It waits for user confirmation before the actual write.
  • Mandatory reconciliation — The agent always runs reconcile_migration after execution, even if the migration failed.
  • execute_rfc guardrail — The agent may use execute_rfc for BAPI interface discovery or verification, but never for bulk data writes. All data loading goes through extract_transform_load.
  • BAPI verification on target — The agent always verifies BAPIs exist on the target system using the target connection_id (for example sap_simulate_rfc(connection_id="<target>", ...)).

​
Common BAPIs

Use caseBAPITable param
Material masterBAPI_MATERIAL_SAVEDATAUses import structures (HEADDATA, CLIENTDATA), not table params — requires per-material calls
Sales ordersBAPI_SALESORDER_CREATEFROMDAT2ORDER_ITEMS_IN
Accounting docsBAPI_ACC_DOCUMENT_POSTACCOUNTGL, CURRENCYAMOUNT
Purchase ordersBAPI_PO_CREATE1POITEM
Business partnersBAPI_BUPA_CREATE_FROM_DATA—
Some BAPIs (like BAPI_MATERIAL_SAVEDATA) use import structures rather than table parameters, which means they process one record per call instead of bulk table input. The agent detects this by verifying the BAPI interface via simulate_rfc before attempting migration.