Queue tasks

To run a worker task on IronWorker, just create a worker object and then use the queue method.

worker = ReportWorker.new
worker.queue

This will send the task off to the IronWorker cloud. It's that simple.

You can set local variables within the worker.

worker = ReportWorker.new
worker.user_id = user_id
worker.queue

As well as loop through and send off many workers at a time. (In this case, we do one username per worker but for simple actions, you may want to include a larger set so as as to amortize the worker setup time.)

usernames.each do |username|
  worker.username = username
  worker.queue
end

You can also queue up workers from within workers. (Make sure you use the merge_worker command so that secondary worker gets uploaded separately.)

class TaskControlWorker < IronWorker::Base

  merge_worker "task_worker", "TaskWorker"

  def run

    @users = User.find(:all)
    @users.each do |user|
      user_worker = TaskWorker.new
      user_worker.user_id = user.id
      user_worker.queue
    end

  end

end

Arguments

[Optional]

• priority: — Priority queue to run the job in (0, 1, 2). p0 is default.
• timeout: — Time in seconds after which to terminate the task.
• recursive — true/false. Default is false. Prevents inadvertent recursion issues.

Priority

IronWorker is multi-tenant system processing thousands and hundreds of thousands of jobs across many users on many servers. IronWorker offers multiple priority levels to make sure you get the best combination of performance/value for your tasks.

Standard priority (p0) is often used for processing thousands of tasks -- crawling or indexing pages -- where some minutes in the queue does not have a material effect.

For tasks that are time-sensitive (such as offloading front-end tasks or when in dev/test mode), we recommend you run in p1 or p2 to run tasks faster.

For example, specifying a higher priority will move tasks ahead of p0 tasks:

Passing a priority of 1 or 2 will cause the job to move through the queue faster. Note that higher priorities cost more, see pricing page for details.

worker.queue(:priority=>1)

worker.queue(:priority=>2)

We recommend using higher priorities for tasks where you'd like an action performed or a response back that's time-critical as well as for test/development. For test/dev you can also use the run_local command especially during initial development/debugging of the worker.

worker.run_local
#worker.queue

Recursive

Workers can schedule and queue other workers (you need to merge them into the worker with the merge_worker command). If you try to queue up a worker that is the same class as the currently running worker, it will be rejected unless you set this explicitly so we know you meant to do it. (Default is false as a protective mechanism.

worker.queue(:recursive=>true)

An example might look like the following:

class RecursiveWorker < IronWorker::Base 

  # No need to merge the worker because it's in the system
  # merge_worker "recursive_worker", "RecursiveWorker" 

  def run 

    new_worker = RecursiveWorker.new 
    new_worker.queue(:recursive=>true) 

  end
end

Timeout

By default, each job has 60 minutes (3600 seconds to complete) after which it is terminated. If you want to set a specific time less than 3600 seconds, pass in an explicit timeout value. In the example, below the task will terminate if it runs longer than 1800 seconds (30 minutes).

worker.queue(:timeout=>1800)

The maximum value for timeout is 3600 seconds. This is to ensure that lots of long running or stalled jobs do not overload the system affecting quality of service. We encourage you to break up long running tasks into separate workers and run them in parallel. Please let us know if you have a reason for a timeout of longer than 60 minutes.

Response

The response from queuing a worker contains a status message, status code, and the task id (within an array stucture).

response = worker.queue

The response looks like the following:

{"msg"=>"Queued up", "status_code"=>200, "tasks"=>[{"id"=>"4ef105519dd7b10d31003ed3"}]}

Wait Until Job Completes

There is also a convenience method worker.wait_until_complete that will wait until the status returned is completed or error.

worker.queue
status = worker.wait_until_complete
puts worker.get_log

You can also call the IronWorker class with the worker.task_id.

status = IronWorker.service.wait_until_complete(worker.task_id)

Note that wait_until_complete is for use with the queue method and not intended for run_local and schedule.

Task ID

You can get the task_id by calling the id method:

worker.task_id

Queuing Workers from a Outside an Application

It's easy to queue up and run worker tasks from outside of an application. This capability is useful in situations when your worker does something that is entirely separate from your application.

Let's say, for example, you want to do some image encoding or something equally as extensive and you don't want the gems, binary code, and other files as dependencies in your Rails app.

To do this, you first create your workers and get them working on IronWorker. After they are working and your code is ready to be run on IronWorker, you can queue it up from a different application by using the name and passing the correct data payload. For instance:

data[:attr_encoded] = Base64.encode64({'@name'=>'Travis'}.to_json)
data[:sw_config] = IronWorker.config.get_atts_to_send
IronWorker.service.queue('HelloWorker', data)