Transit Assistance Program is a reloadable fare card that offers the most convenient and flexible way to pay your fare on Metro bus and rail.
Project Objective
To create a PoC with these requirements: migrate 16TB of data from the TAP database (Oracle) to S3, implement data transformation for 5 tables during migration and generate reports from transformed data then finally visualize it via Tableau and QuickSight.
About the Client
Looking to evaluate the viability of using Amazon Web Services as its next-generation data platform.
Currently, this system maintains a large (approximately 300 TB) data ecosystem around its TAP card, and the volume is expected to grow to over 500 TB soon. Built primarily on Oracle infrastructure, this ecosystem has demonstrated signals when it reaches the upper limit of its capacity to handle the data volumes and transformations required for the every-day business.
Project Scope
Data Migration
Via Data Migration Service, Kinesis Data Streams
Data Transformation
Via AWS Lambda, Glue, Kinesis Firehose
Loading Data
Data after migration will be on S3 or/and Redshift
Data Visualization and Report Generation
Using Athena or other tools to visualize the data and generate reports
Technology
• Project Management: Jira, Confluence • Communication Channels: Slack, MS Teams • AWS Services: RDS, Data Migration Service, Glue, S3, Athena, QuickSight