Skip to content

Latest commit

 

History

History
93 lines (60 loc) · 3.48 KB

readme.md

File metadata and controls

93 lines (60 loc) · 3.48 KB
Raw

Google Cloud Playground

Quickly test Google Cloud services

Prerequisites

Java Dev Env

  • JDK8+
  • Maven

You may also need to set the JAVA_HOME & M2_HOME in the run.sh script.

Google Cloud java client

Github

config file

This is the most important part of this project, most components are configured here.

You can specify which service account to use by simply set the path of the json file to load.

Once it has been configured. Run it :)

./run.sh

Use http proxy for GCP java client libs

http.proxy = on
http.proxy.host = localhost
http.proxy.port = 7443

set it here

BigQuery

refer to this Gist for setting up the testing table.

sample code

Translate API

sample code

Pubsub

This project will create the Topic and the Subscription for you, you can simply turn this off by set

google.pubsub.skip.init = true

in the config file. If you don't do it while the topic and the subscription is there, it will show you a clear runtime exeception and keep running.

You could also use the pubsub component as data generator for testing a streaming system on Google Cloud

sample code

Firebase/GA sample messages to Pubsub

In case you want to produce some Firebase/GA data in json and send to Pubsub, you can use this pubsub module easily.

Key configurations listed below
google.pubsub = on # turn on/off the pubsub module
google.pubsub.skip.init = false # false means create the topic and subscritpion for you
google.pubsub.pub = on # publish the messages to the topic
google.pubsub.sub = on # subscribe the messages and print on the screen for debugging purposes 
google.pubsub.pub.threads = 4
google.pubsub.pub.threads.msgnum = 20
google.pubsub.topic = firebase-rt-topic
google.pubsub.sub.threads = 8
google.pubsub.sub.threads.pulls = 10
google.pubsub.subscription = firebase-rt-sub
firebase.sample.data = /path/to/sample_data.json

Tips:

  • You may skip the init after the first run, or completely turn it off if you have already created the Topic/Sub in some way
  • Turn off the google.pubsub.sub so the following Dataflow/Flink/Spark could consume the messages, unless they have their own subscriptions respectively
  • Get more complexed firebase.sample.data for testing. e.g. select/export random/obfuscated data from existing events table from BigQuery

Pubsublite

sample code. Similar to Pubsub with certain limits

Cloud Storage, aka GCS

sample code