IronWorker Environment & Stack Information

Table of Contents

• Offical Docker Images
• Maximum Data Payload
• Memory per Worker
• Local Disk Space per Worker
• Maximum Run Time per Worker
• Priority Queue Management
• Maximum Scheduled Tasks per Project
• Image Size Limitation
• Scheduled Task Frequency

If you don’t see what you need here, please contact us and tell us what you’re looking for. If it’s a common/popular package, we can certainly look to include it.

Offical Docker Images

To see all of Iron’s offically supported Docker Images, visit us on Docker Hub https://hub.docker.com/r/iron/

Maximum Data Payload

The following is the maximum data payload that can be passed to IronWorker. A data payload that exceeds this size will generate an error response from the API.

Tip: We recommend that you avoid sending large payloads with your workers. Instead use a data store to hold the data and then pass an ID or reference to the worker. The worker can grab the data and then do its processing. It’s more efficient on the API as well as better in terms of creating atomic/stateless processing.

Memory per Worker

The standard worker sandbox environment contains a certain amount of accessible memory. This amount should be sufficient for almost all workloads. We are working on a super worker environment that would allow greater memory allocations. Please contact us if you have specific needs here.

Tip: We recommend distributing workloads over multiple workers—not only for better resource management, but also to take advantage of the massive concurrency enabled by a cloud worker system.

Local Disk Space per Worker

Each worker task has local disk space available to it for use on a temporary basis while the worker is running. You have full read/write privileges to create directories and files inside this space, and can perform most ordinary file operations. This directory is used as the current working directory (“.”) when executing your workers.

Maximum Run Time per Worker

There is a system-wide limit for the maximum length a task may run. Tasks that exceed this limit will be terminated and will have timeout as their status.

Tip: You should design your tasks to be moderate in terms of the length of time they take to run. If operations are small in nature (seconds or milliseconds) then you’ll want to group them together so as to amortize the worker setup costs. Likewise, if they are long-running operations, you should break them up into a number of workers. Note that you can chain together workers as well as use IronMQ, scheduled jobs, and datastores to orchestrate a complex series or sequence of tasks.

Priority Queue Management

Each priority (p0, p1, p2) has a targeted maximum time limit for tasks sitting in the queue. Average queue times will typically be less than those listed on the pricing page. High numbers of tasks, however, could raise those average queue times for all users. To keep the processing time for high priority jobs down, per user capacities are in place for high priority queues. Limits are on per-queue basis and are reset hourly. High priority tasks that exceed the limit, are queued at the next highest priority. Only under high overall system load should queue times for tasks exceeding the capacity extend beyond the initial targeted time limits.

Priority	Capacity Per Hour Per User
p2	100
p1	250

Maximum Scheduled Tasks per Project

The following is the default number of scheduled tasks. It should be sufficient for even the largest projects. If you would like this number increased, however, please feel free to contact us.

Tip: A common mistake is to create scheduled jobs on a per user or per item basis. Instead, use scheduled jobs as master tasks that orchestrate activities around sets of users or items. When scheduled tasks run, they can access databases to get a list of actions to perform and then queue up one or more workers to handle the set. View the page on scheduling for more information on scheduling patterns and best practices.

Image Size Limitation

There is a limit for the image size. Image that exceeds this limitation will cause an error. Please contact us if you have specific needs.

Tip: Review your code and remove all the dependencies that will not be used. Also using Alpine Linux as a base docker image, could help to decrease image size compared to traditional Linux distros.

Scheduled Task Frequency

Tasks can be scheduled to run every N seconds or more specifying N using the run_every parameter and where N > 60. (The minimum frequency is every 60 seconds.)