Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

impl(batch/cpp_application): add a cloud batch example for a cpp application #327

Merged
merged 4 commits into from
May 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ cmake-build-debug/
.build/
.vscode/
cpp-samples-checkers/
build/
1 change: 1 addition & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ include(ExternalProject)

set(samples
# cmake-format: sort
batch/cpp_application
batch/simple
bigquery/write
cloud-run-hello-world
Expand Down
28 changes: 28 additions & 0 deletions batch/cpp_application/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# ~~~
# Copyright 2024 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ~~~

cmake_minimum_required(VERSION 3.20)
set(CMAKE_CXX_STANDARD 20)

# Define the project name and where to report bugs.
set(PACKAGE_BUGREPORT
"https://github.com/GoogleCloudPlatform/cpp-samples/issues")
project(cpp-samples-batch CXX)

find_package(google_cloud_cpp_batch REQUIRED)

add_executable(driver driver.cc)
target_link_libraries(driver PRIVATE google-cloud-cpp::batch)
169 changes: 169 additions & 0 deletions batch/cpp_application/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
# Using Cloud Batch

This example shows how to run a C++ application on Cloud Batch job using C++.

If you are not familiar with the Batch API, we recommend you first read the
[API overview] before starting this guide.

## The example

The following steps are included:

1. Create a docker image
1. Upload it to Artifact registry
1. Create the job
1. Poll until the job finishes

## Pre-reqs

1. Install the [gcloud CLI](https://cloud.google.com/sdk/docs/install).
1. Install [docker](https://docs.docker.com/engine/install/).

## 1. Create the docker image

The instructions are [here](application/README.md).

## 2. Upload it to Artifact registry

1. \[If it does not already exist\] Create the artifact repository
1. Build the image locally
1. Tag and push the image to the artifact repository

### 1. Create the artifact repository

To run this example, replace the `[PROJECT ID]` placeholder with the id of your
project:

Authorize via gcloud cli

```shell
PROJECT_ID=[PROJECT_ID]
LOCATION="us-central1"
REPOSITORY="application-repo"

gcloud auth login
gcloud config set project ${PROJECT_ID}
# Create the repository
gcloud artifacts repositories create ${REPOSITORY} \
--repository-format=docker \
--location=${LOCATION} \
--description="Store the example C++ application" \
--async
```

<details>
<summary>To verify repo was created</summary>
```
gcloud artifacts repositories list
```

You should see something like

```
application-repo DOCKER STANDARD_REPOSITORY Store the example C++ application us-central1 Google-managed key 2024-05-13T20:07:11 2024-05-13T20:07:11 0
```

</details>

### 2. Build the image locally

```
cd batch/cpp_application/application
docker build --tag=application-image:latest .
```

### 3. Tag and push the image to the artifact repository

```
cd batch/cpp_application/application
gcloud builds submit --region=${LOCATION} --tag ${LOCATION}-docker.pkg.dev/${PROJECT_ID}/${REPOSITORY}/application-image:latest
```

<details>
<summary>Using docker</summary>
To do the same using docker instead of the gcloud CLI:

```
# Tag the image
docker tag application-image:latest ${LOCATION}-docker.pkg.dev/${PROJECT_ID}/${REPOSITORY}/application-image:latest

# Push the image
docker push ${LOCATION}-docker.pkg.dev/${PROJECT_ID}/${REPOSITORY}/application-image:latest
```

</details>

## 3. Create the job
alevenberg marked this conversation as resolved.
Show resolved Hide resolved

You can do either of the following:

1. Use the C++ client libraries to create and poll for the job until completion
1. Use the gcloud CLI to create the job

### Using the C++ Client libraries

#### Compiling the Example

This project uses `vcpkg` to install its dependencies. Clone `vcpkg` in your
`$HOME`:

```shell
git clone -C $HOME https://github.com/microsoft/vcpkg.git
```

Install the typical development tools, on Ubuntu you would use:

```shell
apt update && apt install -y build-essential cmake git ninja-build pkg-config g++ curl tar zip unzip
```

In this directory compile the dependencies and the code, this can take as long
as an hour, depending on the performance of your workstation:

```shell
cd cpp-samples/batch/cpp_application
cmake -S . -B .build -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_TOOLCHAIN_FILE=$HOME/vcpkg/scripts/buildsystems/vcpkg.cmake
cmake --build .build
```

#### Run the sample

Run the example, replace the `[PROJECT ID]` placeholder with the id of your
project:

```shell
.build/driver [PROJECT ID] us-central1 cpp-application-run application.json application-repo
```

This submits the batch job and then polls until the job is complete.

### Using the gcloud CLI

1. Replace the `imageURI` field in application.json

```
"runnables": [
{
"container": {
"imageUri": "${LOCATION_ID}-docker.pkg.dev/${PROJECT_ID}/{REPOSITORY}/application-image:latest",
}
}
],
```

2. Submit the job

```
gcloud batch jobs submit cpp-application-cli-run \
--config=application.json \
--location=us-central1
```

3. Check on the job status

```
gcloud batch jobs describe cpp-application-cli-run --location=us-central1
```

[api overview]: https://cloud.google.com/batch/docs
36 changes: 36 additions & 0 deletions batch/cpp_application/application.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
{
"taskGroups": [
{
"taskSpec": {
"runnables": [
{
"container": {
"imageUri": "",
}
}
],
"computeResource": {
"cpuMilli": 2000,
"memoryMib": 16
},
"maxRetryCount": 2,
"maxRunDuration": "3600s"
},
"taskCount": 1,
"parallelism": 1
}
],
"allocationPolicy": {
"instances": [
{
"policy": { "machineType": "e2-standard-4" }
}
]
},
"labels": {
"env": "testing"
},
"logsPolicy": {
"destination": "CLOUD_LOGGING"
}
}
24 changes: 24 additions & 0 deletions batch/cpp_application/application/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# ~~~
# Copyright 2024 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ~~~

cmake_minimum_required(VERSION 3.20)

# Define the project name and where to report bugs.
set(PACKAGE_BUGREPORT
"https://github.com/GoogleCloudPlatform/cpp-samples/issues")
project(cpp-samples-batch CXX)

add_executable(main src/main.cc)
51 changes: 51 additions & 0 deletions batch/cpp_application/application/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Copyright 2024 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# We chose Alpine to build the image because it has good support for creating
# statically-linked, small programs.
FROM alpine:3.19 AS build

# Install the typical development tools for C++, and
# the base OS headers and libraries.
RUN apk update && \
apk add \
build-base \
cmake \
curl \
git \
gcc \
g++ \
libc-dev \
linux-headers \
ninja \
pkgconfig \
tar \
unzip \
zip

# Copy the source code to /src and compile it.
COPY . /src
WORKDIR /src

# Run the CMake configuration step, setting the options to create
# a statically linked C++ program
RUN cmake -S /src -B /build -GNinja \
-DCMAKE_BUILD_TYPE=Release

# Compile the binary and strip it to reduce its size.
RUN cmake --build /build
RUN strip /build/main

# Make the program the entry point.
ENTRYPOINT [ "/build/main" ]
21 changes: 21 additions & 0 deletions batch/cpp_application/application/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# To build and run

```
cmake -S . -B build
cmake --build build
build/main
```

# To create docker image

```
docker build --tag=application-image:latest .
```

## To run and enter your image

```
docker run -it --entrypoint bash application-image:latest
```

To exit container, type `exit`
17 changes: 17 additions & 0 deletions batch/cpp_application/application/src/main.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
// Copyright 2024 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

#include <iostream>

int main(int argc, char* argv[]) { std::cout << "Running my application\n"; }
Loading