(A “batch” in this context is the number of items Lambda will pull at one time. If your Lambdas are self-contained and can scale to the moon, great! By default, Lambda will pull up to 5 batches of events from the queue at the same time. But with a serverless approach, you’d have one method that retrieves all your data from the database and stuffs each record into a queue then your serverless app would pull items from the queue and process them individually.ĪWS Lambda can be configured to pull from an SQS queue, so it’s a beautiful and nearly bulletproof way to process lots of data. In the olden days, if you had 5,000 events to process every day, you might have a single long-running process that pulls all records from the database in a single query, then loops through the results and processes each in one big process. This saves money, and effectively eliminates setting up autoscaling.īecause serverless functions are generally intended to be short-running, you’ll want to make your serverless function process just one event at a time. Instead of maintaining servers sitting idly when they’re not needed, a serverless app can scale from zero to the moon, then scale back down to zero. One thing serverless apps can be great at is batch processing, where you might have thousands of events to process, but you only need to do this daily or weekly. Queues and batch processing in serverless apps We’ll try to keep it approachable, but it’s gonna be a ride. Note: What follows gets a little technical, and is highly specific to Lambda (AWS’s serverless computing platform) and SQS (AWS’s queuing system). We’ve also got some serverless starter packages for Netlify and Cloudflare (for a Cloudflare-based membership protected site, for instance). We use serverless apps internally for a number of things, like WAF automation (to block card testers), our webhooks system, our new Help Scout + Foxy ecommerce integration, and the custom shipping code functionality within Foxy. We’re big fans of serverless here at Foxy.io, both in general, and the framework specifically.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |