Why AWS Batch is Blessing for some and curse for others
Rating: 5 out of 10
IncentivizedUse Cases and Deployment Scope
We encountered issues orchestrating long-running tasks in Lambda functions, as they have a limit of 15 minutes. We needed a solution that could be triggered from AWS Step Functions. Then we found yes it's a bit tedious to set up, has a steep learning curve but once that's passed you can easily run a 24 or 48 hours or other long-timed task without any issues, which helped our use case where needed to run a task which took 24 hours even with AWS Batch's Compute
Pros
- Easy to orchestrate and trigger jobs
- No time limit issues like lambda
- Multiple Jobs can be run in same single compute and job queue
- JOb queue can queue up task for parralled or serialization
Cons
- Steep Learning Curve
- Setting up jobs could be easy
- Giving roles access to compute environment is hard and confusing
Likelihood to Recommend
In our case, we had a SQL shell script that ingested data into RDS using the MySQL command-line interface (CLI). However, the file was 4 gigabytes in size, so reading each line, transforming the data, and writing it to different tables for various files took a considerable amount of time.
Less Appropriate: if your job doesn't run on demand or can be completed in less than 15 mins , batch doesn't set your use case as the steep learning complexity for such a small time consuming workload is better spent on lambdas or EC2 configuration as well , and if you don't know but batch is a lot more expensive than lambda's or basic EC2
Less Appropriate: if your job doesn't run on demand or can be completed in less than 15 mins , batch doesn't set your use case as the steep learning complexity for such a small time consuming workload is better spent on lambdas or EC2 configuration as well , and if you don't know but batch is a lot more expensive than lambda's or basic EC2