Automate Google Cloud Dataproc
HumDay reads Google Cloud Dataproc’s own published API description and derives 34 operations from it. Describe the outcome you want in plain words — you get a program that is written, proven on real data, and run for you.
What Google Cloud Dataproc is
Manages Hadoop-based clusters and jobs on Google Cloud Platform.
API host: dataproc.googleapis.com
What HumDay can do in Google Cloud Dataproc
26 documented operations change something in Google Cloud Dataproc.
- POSTCreates new workflow template.
/v1/{parent}/workflowTemplates - POSTCreates new autoscaling policy.
/v1/{parent}/autoscalingPolicies - POSTCreates a batch workload that executes asynchronously.
/v1/{parent}/batches - POSTCreates a node group in a cluster.
/v1/{parent}/nodeGroups - DELETEDeletes a workflow template.
/v1/{name} - PUTUpdates (replaces) workflow template.
/v1/{name} - POSTStarts asynchronous cancellation on a long-running operation.
/v1/{name}:cancel - POSTResizes a node group in a cluster.
/v1/{name}:resize - POSTInstantiates a template and begins execution.The returned Operation can be used to track execution of workflow by polling operations.get.
/v1/{name}:instantiate - POSTGets the access control policy for a resource.
/v1/{resource}:getIamPolicy - POSTSets the access control policy on the specified resource.
/v1/{resource}:setIamPolicy - POSTReturns permissions that a caller has on the specified resource.
/v1/{resource}:testIamPermissions - POSTInject encrypted credentials into all of the VMs in a cluster.The target cluster must be a personal auth cluster assigned to the user who is issuing the RPC.
/v1/{project}/{region}/{cluster}:injectCredentials - POSTInstantiates a template and begins execution.This method is equivalent to executing the sequence CreateWorkflowTemplate, InstantiateWorkflowTemplate, DeleteWorkflowTemplate.The re…
/v1/{parent}/workflowTemplates:instantiateInline - POSTCreates a cluster in a project.
/v1/projects/{projectId}/regions/{region}/clusters - POSTSubmits a job to a cluster.
/v1/projects/{projectId}/regions/{region}/jobs:submit - POSTSubmits job to a cluster.
/v1/projects/{projectId}/regions/{region}/jobs:submitAsOperation - DELETEDeletes the job from the project.
/v1/projects/{projectId}/regions/{region}/jobs/{jobId} - PATCHUpdates a job in a project.
/v1/projects/{projectId}/regions/{region}/jobs/{jobId} - POSTStarts a job cancellation request.
/v1/projects/{projectId}/regions/{region}/jobs/{jobId}:cancel - DELETEDeletes a cluster in a project.
/v1/projects/{projectId}/regions/{region}/clusters/{clusterName} - PATCHUpdates a cluster in a project.
/v1/projects/{projectId}/regions/{region}/clusters/{clusterName} - POSTStops a cluster in a project.
/v1/projects/{projectId}/regions/{region}/clusters/{clusterName}:stop - POSTStarts a cluster in a project.
/v1/projects/{projectId}/regions/{region}/clusters/{clusterName}:start - POSTRepairs a cluster.
/v1/projects/{projectId}/regions/{region}/clusters/{clusterName}:repair - POSTGets cluster diagnostic information.
/v1/projects/{projectId}/regions/{region}/clusters/{clusterName}:diagnose
What HumDay can read from Google Cloud Dataproc
These are the operations a schedule or a trigger can watch.
- GETRetrieves the latest workflow template.Can retrieve previously instantiated template by specifying optional version parameter.
/v1/{name} - GETLists batch workloads.
/v1/{parent}/batches - GETLists workflows that match the specified filter in the request.
/v1/{parent}/workflowTemplates - GETLists autoscaling policies in the project.
/v1/{parent}/autoscalingPolicies - GETLists regions/{region}/jobs in a project.
/v1/projects/{projectId}/regions/{region}/jobs - GETLists all regions/{region}/clusters in a project alphabetically.
/v1/projects/{projectId}/regions/{region}/clusters - GETGets the resource representation for a job in a project.
/v1/projects/{projectId}/regions/{region}/jobs/{jobId} - GETGets the resource representation for a cluster in a project.
/v1/projects/{projectId}/regions/{region}/clusters/{clusterName}
How automating Google Cloud Dataproc works
- Describe the outcome. Say what you want to happen, in your own words. No node graphs, no field mapping.
- Approve the contract. HumDay writes down exactly what it will do, what it will touch, and what it will never do. You approve it before anything is built.
- See it proven. The program runs and shows you the result before it is allowed near your live Google Cloud Dataproc account.
- Grant access, then go live. You approve the specific Google Cloud Dataproc operations it may use — and only those.
Automate Google Cloud Dataproc with these
- Gmail79 operations
- Google+9 operations
- Google Abusive Experience Report2 operations
- Google Accelerated Mobile Pages (AMP) URL1 operations
- Google Access Approval7 operations
- Google Access Context Manager9 operations
- Google ACME DNS2 operations
- Google Ad Exchange Buyer38 operations
- Google Ad Exchange Buyer II50 operations
- Google Ad Experience Report2 operations
- Google Admin SDK122 operations
- Google AdMob7 operations
Categories
Questions about Google Cloud Dataproc automation
- Can HumDay connect to Google Cloud Dataproc?
- Yes. HumDay reads Google Cloud Dataproc's own published API description and derives the operations from it, so there is no hand-built connector to wait for. 34 operations are documented.
- Do I need to write code to automate Google Cloud Dataproc?
- No. You describe the outcome you want in plain words. HumDay agrees a contract with you, writes the program, and shows you a test run before anything touches your Google Cloud Dataproc account.
- What can HumDay do in Google Cloud Dataproc?
- 26 of the 34 documented operations change something in Google Cloud Dataproc, and 8 read from it. HumDay only ever uses the specific operations your approved contract needs.
- Is my Google Cloud Dataproc account safe?
- Your credentials are stored encrypted and are never shown in chat, code, or logs. Every run is limited to the operations you explicitly approved, and anything that writes to Google Cloud Dataproc is held behind that approval.
Where this came from
The operations above are read from a published API description for Google Cloud Dataproc at dataproc.googleapis.com/$discovery/rest?version=v1. Descriptions are the provider’s own words, not ours. Last published 2023-04-21.