IronWorker Environment & Stack Information
Table of Contents
If you don't see what you need here, please contact us and tell us what you're looking for. If it's a common/popular package, we can certainly look to include it.
Maximum Data Payload
The following is the maximum data payload that can be passed to IronWorker. A data payload that exceeds this size will generate an error response from the API.
Tip: We recommend that you avoid sending large payloads with your workers. Instead use a data store to hold the data and then pass an ID or reference to the worker. The worker can grab the data and then do its processing. It's more efficient on the API as well as better in terms of creating atomic/stateless processing.
Memory per Worker
The standard worker sandbox environment contains a certain amount of accessible memory. This amount should be sufficient for almost all workloads. We are working on a super worker environment that would allow greater memory allocations. Please contact us if you have specific needs here.
Tip: We recommend distributing workloads over multiple workers—not only for better resource management, but also to take advantage of the massive concurrency enabled by a cloud worker system.
Local Disk Space per Worker
Each worker task has local disk space available to it for use on a temporary basis while the worker is running. You have full read/write privileges to create directories and files inside this space, and can perform most ordinary file operations. This directory is used as the current working directory (".") when executing your workers.
Maximum Run Time per Worker
There is a system-wide limit for the maximum length a task may run. Tasks that exceed this limit will be terminated and will have
timeout as their status.
Tip: You should design your tasks to be moderate in terms of the length of time they take to run. If operations are small in nature (seconds or milliseconds) then you'll want to group them together so as to amortize the worker setup costs. Likewise, if they are long-running operations, you should break them up into a number of workers. Note that you can chain together workers as well as use IronMQ, scheduled jobs, and datastores to orchestrate a complex series or sequence of tasks.
Priority Queue Management
Each priority (p0, p1, p2) has a targeted maximum time limit for tasks sitting in the queue. Average queue times will typically be less than those listed on the pricing page. High numbers of tasks, however, could raise those average queue times for all users. To keep the processing time for high priority jobs down, per user capacities are in place for high priority queues. Limits are on per-queue basis and are reset hourly. High priority tasks that exceed the limit, are queued at the next highest priority. Only under high overall system load should queue times for tasks exceeding the capacity extend beyond the initial targeted time limits. Usage rates will be based on the actual priority tasks run on, not the priority initially queued.
|Priority||Capacity Per Hour Per User|
Maximum Scheduled Tasks per Project
The following is the default number of scheduled tasks. It should be sufficient for even the largest projects. If you would like this number increased, however, please feel free to contact us.
Tip: A common mistake is to create scheduled jobs on a per user or per item basis. Instead, use scheduled jobs as master tasks that orchestrate activities around sets of users or items. When scheduled tasks run, they can access databases to get a list of actions to perform and then queue up one or more workers to handle the set. View the page on scheduling for more information on scheduling patterns and best practices.
Scheduled Task Frequency
Tasks can be scheduled to run every N seconds or more specifying N using the
run_every parameter and where N > 60. (The minimum frequency is every 60 seconds.)
Note: A task may be executed a short time after its scheduled frequency depending on the priority level. (Scheduled tasks can be given a priority; higher priorities can reduce the maximum time allowed in queue.)
Security Groups and IP Ranges
IronWorker provides an AWS security group and IP ranges in the event users want to isolate AWS EC2, RDS, or other services to these groups/ranges.
|EC2 Security Group||Account ID||Security Group String|