What I built
- I owned the project end to end, from analyzing the source data to loading it into the company environment.
- Automatic extraction: replaced the manual production of the 100+ monthly reports and made the process repeatable, with no human effort needed.
- Cleaning and treatment: cross-referenced the legacy data with the management system to fix wrong information, recover unreadable records and enrich incomplete ones before loading.
Under the hood
Data checked before it is released
Every new load is validated by comparing totals and samples against the source. Only when they match does the data reach the people who will use it.
No strain on the production system
Extraction queries go through performance analysis and run outside peak hours, so nobody notices the pipeline working.
A process that repeats itself
A scheduler handles orchestration and logs every run. When a business rule changes, the fix is made in a single place.
The result
- The data came under the control of the company, and the dependence on the vendor ended.
- Estimated savings of R$ 2 million in contract fees.