appengine-ruby-rails-activejob-pubsub
Learn how to run background jobs using Ruby on Rails ActiveJob.
This tutorial shows how to create and configure a Ruby on Rails application to run background processing jobs on App Engine flexible environment using ActiveJob and Cloud Pub/Sub.

Objectives

    Create a background processing job
    Deploy your application to App Engine flexible environment
    Verify background jobs are running

Before you begin

You'll need the following:

Costs

This tutorial uses billable components of Cloud Platform including:
    App Engine flexible environment
    Cloud Pub/Sub
Use the pricing calculator to generate a cost estimate based on your projected usage. New GCP users might be eligible for a free trial.

Creating your background job

Starting with version 4.2, Ruby on Rails includes an abstraction layer around background job processing called ActiveJob. This abstraction allows you to write the queuing and execution logic independently of queue and job runner implementations.
Ruby on Rails provides command-line tools for generating templated skeletons for things such as database migrations, controllers, and even background jobs.
You will create a job named HelloJob that will accept a name argument and print "Hello #{name}" to standard output.
    1.
    Use the Rails generator feature to create HelloJob:
    1
    bin/rails generate job Hello
    Copied!
    Rails creates stub files from templates:
    1
    invoke test_unit
    2
    create test/jobs/hello_job_test.rb
    3
    create app/jobs/hello_job.rb
    4
    create app/jobs/application_job.rb
    Copied!
    2.
    Edit your app/jobs/hello_job.rb with the following:
    1
    class HelloJob < ApplicationJob
    2
    queue_as :default
    3
    ​
    4
    def perform(name)
    5
    # Do something later
    6
    puts "Hello, #{name}"
    7
    end
    8
    end
    Copied!

Create a test URL to queue the job

You will create a controller named HelloController that will provide an action called say which will queue our HelloJob to execute in the background.
    1.
    Use the Rails generator feature to create HelloController:
    1
    bin/rails generate controller Hello
    Copied!
    Rails creates stub files from templates:
    1
    create app/controllers/hello_controller.rb
    2
    invoke erb
    3
    create app/views/hello
    4
    invoke test_unit
    5
    create test/controllers/hello_controller_test.rb
    6
    invoke helper
    7
    create app/helpers/hello_helper.rb
    8
    invoke test_unit
    9
    invoke assets
    10
    invoke coffee
    11
    create app/assets/javascripts/hello.coffee
    12
    invoke scss
    13
    create app/assets/stylesheets/hello.scss
    Copied!
    2.
    Add a say action to HelloController. Edit your app/controllers/hello_controller.rb with the following:
    1
    class HelloController < ApplicationController
    2
    def say
    3
    HelloJob.perform_later(params[:name])
    4
    render plain: 'OK'
    5
    end
    6
    end
    Copied!
    This action will queue our HelloJob with the provided request parameter name.
    3.
    Create a route to this action. In config/routes.rb, add:
    1
    get '/hello/:name', to: 'hello#say'
    Copied!
    When you make an HTTP GET request to /hello/Jeff, the HelloController will handle the request using the say action with parameter :name as "Jeff"

Configuring your background worker to use Cloud Pub/Sub

ActiveJob can be configured with various different background job runners. This tutorial will cover ActiveJob::GoogleCloudPubsub which uses Cloud Pub/Sub to manage the job queue.
    1.
    Add activejob-google_cloud_pubsub gem to your Gemfile:
    1
    bundle add activejob-google_cloud_pubsub
    Copied!
    2.
    Configure ActiveJob to use GoogleCloudPubsub as its queue adapter. In config/application.rb:
    1
    class Application < Rails::Application
    2
    # ...
    3
    config.active_job.queue_adapter = :google_cloud_pubsub
    4
    end
    Copied!

Deploying to App Engine flexible environment

Option A: Shared worker and web application

For this option, the App Engine service will run both the web server and a worker process via a process manager called foreman. If you choose this method, App Engine will scale your web and worker instances together.
    1.
    Add foreman gem to your Gemfile:
    1
    bundle add foreman
    Copied!
    2.
    Create a Procfile at the root of your application:
    1
    web: bundle exec rails server -p 8080
    2
    worker: bundle exec activejob-google_cloud_pubsub-worker
    Copied!
    3.
    Create an app.yaml for deploying the application to App Engine:
    1
    runtime: ruby
    2
    env: flex
    3
    ​
    4
    entrypoint: bundle exec foreman start
    5
    ​
    6
    env_variables:
    7
    SECRET_KEY_BASE: [SECRET_KEY]
    Copied!
    Be sure to replace the [SECRET_KEY] with a secret key for Rails sessions.
    4.
    Deploy to App Engine
    1
    gcloud app deploy app.yaml
    Copied!

Option B: Separate worker and web application

For this option, you are creating 2 App Engine services - one runs the web server and one runs worker processes. Both services use the same application code. This configuration allows you to scale background worker instances independently of your web instances at the cost of potentially using more resources.
    1.
    Create an app.yaml for deploying the web service to App Engine:
    1
    runtime: ruby
    2
    env: flex
    3
    ​
    4
    entrypoint: bundle exec rails server -p 8080
    5
    ​
    6
    env_variables:
    7
    SECRET_KEY_BASE: [SECRET_KEY]
    Copied!
    Be sure to replace the [SECRET_KEY] with a secret key for Rails sessions.
    2.
    Create a worker.yaml for deploying the worker service to App Engine:
    1
    runtime: ruby
    2
    env: flex
    3
    service: worker
    4
    ​
    5
    entrypoint: bundle exec activejob-google_cloud_pubsub-worker
    6
    ​
    7
    env_variables:
    8
    SECRET_KEY_BASE: [SECRET_KEY]
    9
    ​
    10
    # Optional scaling configuration
    11
    manual_scaling:
    12
    instances: 1
    Copied!
    Be sure to replace the [SECRET_KEY] with a secret key for Rails sessions.
    Note that the health check is disabled here because the worker service is not running a web server and cannot respond to the health check ping.
    As mentioned above, you can configure scaling for the worker service independently of the default (web) service. In the manual_scaling section, you have configured the worker service to start with 1 worker instance. For more information on scaling options, see scaling configuration options in app.yaml. If you choose an automatic_scaling option, be aware that scaling for the background processing is based off of CPU utilization, not queue size.
    3.
    Deploy both services to App Engine
    1
    gcloud app deploy app.yaml worker.yaml
    Copied!

Verify your background queuing works

    1.
    In the Cloud Platform Console, go to the App Engine Services page. Locate the service that is running your background workers (if option A, it should be the default service, if option B, it should be the worker service). Click Tools -> Logs for that service.
    2.
    In a separate window, navigate to your deployed Rails application at:
    1
    https://[YOUR_PROJECT_ID].appspot.com/hello/Jeff
    Copied!
    Be sure to replace [YOUR_PROJECT_ID] with your Google Cloud Platform project ID.
    3.
    Navigate back to the Logs dashboard. In the Filter by label or text search field, add "Hello, Jeff" and you should see a logging statement like the following if using foreman:
    1
    13:13:52.000 worker.1 | Hello, Jeff
    Copied!
    or if using a second service:
    1
    13:13:52.000 Hello, Jeff
    Copied!
Congratulations, you have successfully set up background job processing on App Engine with Cloud Pub/Sub.

Cleaning up

After you've finished this tutorial, you can clean up the resources you created on Google Cloud Platform so you won't be billed for them in the future. The following sections describe how to delete or turn off these resources.

Deleting the project

The easiest way to eliminate billing is to delete the project you created for the tutorial.
To delete the project:
    1.
    In the Cloud Platform Console, go to the Projects page.
    2.
    Click the trash can icon to the right of the project name.
Warning: Deleting a project has the following consequences:
If you used an existing project, you'll also delete any other work you've done in the project. You can't reuse the project ID of a deleted project. If you created a custom project ID that you plan to use in the future, you should delete the resources inside the project instead. This ensures that URLs that use the project ID, such as an appspot.com URL, remain available.

Deleting App Engine services

To delete an App Engine service:
    1.
    In the Cloud Platform Console, go to the App Engine Services page.
    2.
    Click the checkbox next to the service you wish to delete.
    3.
    Click Delete at the top of the page to delete the service.
If you are trying to delete the default service, you cannot. Instead:
    1.
    Click on the number of versions which will navigate you to the App Engine Versions page.
    2.
    Select all the versions you wish to disable and click Stop at the top of the page. This will free
    all of the Google Compute Engine resources used for this App Engine service.

Next steps

Last modified 7mo ago