Showing posts with label GKE. Show all posts
Showing posts with label GKE. Show all posts

Deploy an app in a container image to a GKE cluster

  •  Create a Hello World app.
  • Package the app into a container image using Cloud Build.
  • Create a cluster in Google Kubernetes Engine (GKE).
  • Deploy the container image to your cluster.


ketan_patel@cloudshell:~ (svo-mvp)$ gcloud container clusters get-credentials batterygke1 --region us-central1 --project svo-mvp




Writing the sample app (Creating Hellow World app using "Go")




ketan_patel@cloudshell:~ (svo-mvp)$ mkdir helloworld-gke
ketan_patel@cloudshell:~ (svo-mvp)$ cd helloworld-gke/


Create a new module named example.com/helloworld:



ketan_patel@cloudshell:~/helloworld-gke (svo-mvp)$ go mod init example.com/helloworld
go: creating new go.mod: module example.com/helloworld
ketan_patel@cloudshell:~/helloworld-gke (svo-mvp)$ ls
go.mod

Create a new file named helloworld.go and paste the following code into it:



ketan_patel@cloudshell:~/helloworld-gke (svo-mvp)$ pwd
/home/ketan_patel/helloworld-gke
ketan_patel@cloudshell:~/helloworld-gke (svo-mvp)$ vi helloworld.go

This code creates a web server that listens on the port defined by the PORT environment variable.


Your app is finished and ready to be packaged in a Docker container, and then uploaded to Artifact Registry.

ketan_patel@cloudshell:~/helloworld-gke (svo-mvp)$ cat helloworld.go 

package main

import (
        "fmt"
        "log"
        "net/http"
        "os"
)

func main() {
        http.HandleFunc("/", handler)

        port := os.Getenv("PORT")
        if port == "" {
                port = "8080"
        }


        log.Printf("Listening on localhost:%s", port)
        log.Fatal(http.ListenAndServe(fmt.Sprintf(":%s", port), nil))
}

func handler(w http.ResponseWriter, r *http.Request) {
        log.Print("Hello world received a request.")
        target := os.Getenv("TARGET")
        if target == "" {
                target = "World"
        }
        fmt.Fprintf(w, "Hello %s!\n", target)
}
ketan_patel@cloudshell:~/helloworld-gke (svo-mvp)$ vi Dockerfile



Containerizing an app with Cloud Build

To containerize the sample app, create a new file named Dockerfile in the same directory as the source files, and copy the following content:


ketan_patel@cloudshell:~/helloworld-gke (svo-mvp)$ cat Dockerfile 


# This is based on Debian and sets the GOPATH to /go.
# https://hub.docker.com/_/golang

FROM golang:1.19.2 as builder
WORKDIR /app

# Initialize a new Go module.
RUN go mod init quickstart-go

# Copy local code to the container image.
COPY *.go ./

# Build the command inside the container.
RUN CGO_ENABLED=0 GOOS=linux go build -o /quickstart-go

# Use a Docker multi-stage build to create a lean production image.
# https://docs.docker.com/develop/develop-images/multistage-build/#use-multi-stage-builds
FROM gcr.io/distroless/base-debian11

# Change the working directory.
WORKDIR /

# Copy the binary to the production image from the builder stage.
COPY --from=builder /quickstart-go /quickstart-go

# Run the web service on container startup.
USER nonroot:nonroot
ENTRYPOINT ["/quickstart-go"]


Get your Google Cloud project ID:


ketan_patel@cloudshell:~/helloworld-gke (svo-mvp)$ gcloud config get-value project
Your active configuration is: [cloudshell-28830]
svo-mvp


Store your container in Artifact Registry and deploy it to your cluster from the registry. 


Run the following command to create a repository named hello-repo in the same location as your cluster:

ketan_patel@cloudshell:~/helloworld-gke (svo-mvp)$ gcloud artifacts repositories create hello-repo \
    --project=svo-mvp \                                                                                                                                          
    --repository-format=docker \
    --location=us-central1 \
    --description="Docker repository"

Create request issued for: [hello-repo]
Waiting for operation [projects/svo-mvp/locations/us-central1/operations/4fb48d80-4e34-4b7f-a0cc-2f716d9727c5] to complete...done.                              
Created repository [hello-repo].


Build your container image using Cloud Build, which is similar to running docker build and docker push, but the build happens on Google Cloud:


ketan_patel@cloudshell:~/helloworld-gke (svo-mvp)$ gcloud builds submit \
  --tag us-central1-docker.pkg.dev/svo-mvp/hello-repo/helloworld-gke .

Creating temporary tarball archive of 3 file(s) totalling 2.1 KiB before compression.
Uploading tarball of [.] to [gs://svo-mvp_cloudbuild/source/1690835076.632607-cf8690a62ffa48eba3f560f17aa9ae4f.tgz]
Created [https://cloudbuild.googleapis.com/v1/projects/svo-mvp/locations/global/builds/4cd4f29d-f1f0-49cb-a01a-55d9026684dd].
Logs are available at [ https://console.cloud.google.com/cloud-build/builds/4cd4f29d-f1f0-49cb-a01a-55d9026684dd?project=180636258465 ].
---------------------------------------------------------------------------------- REMOTE BUILD OUTPUT -----------------------------------------------------------------------------------
starting build "4cd4f29d-f1f0-49cb-a01a-55d9026684dd"

FETCHSOURCE
Fetching storage object: gs://svo-mvp_cloudbuild/source/1690835076.632607-cf8690a62ffa48eba3f560f17aa9ae4f.tgz#1690835078050696
Copying gs://svo-mvp_cloudbuild/source/1690835076.632607-cf8690a62ffa48eba3f560f17aa9ae4f.tgz#1690835078050696...
/ [1 files][  1.4 KiB/  1.4 KiB]                                                
Operation completed over 1 objects/1.4 KiB.
BUILD
Already have image (with digest): gcr.io/cloud-builders/docker
Sending build context to Docker daemon  5.632kB
Step 1/10 : FROM golang:1.19.2 as builder
1.19.2: Pulling from library/golang
17c9e6141fdb: Already exists
de4a4c6caea8: Already exists
d0a75b47d936: Pull complete
Digest: sha256:992d5fea982526ce265a0631a391e3c94694f4d15190fd170f35d91b2e6cb0ba
Status: Downloaded newer image for golang:1.19.2
 dce494d5814b
Step 2/10 : WORKDIR /app
 Running in ddd37497cbac
Removing intermediate container ddd37497cbac
 b2b056d797bc
Step 3/10 : RUN go mod init quickstart-go
 Running in d42bc942428d
go: creating new go.mod: module quickstart-go
Removing intermediate container d42bc942428d
 42c8c7f54f0a
Step 4/10 : COPY *.go ./
 b0e902994cc7
Step 5/10 : RUN CGO_ENABLED=0 GOOS=linux go build -o /quickstart-go
 Running in b27996dde697
Removing intermediate container b27996dde697
 68da3043b929
Step 6/10 : FROM gcr.io/distroless/base-debian11
latest: Pulling from distroless/base-debian11
Digest: sha256:73deaaf6a207c1a33850257ba74e0f196bc418636cada9943a03d7abea980d6d
Status: Downloaded newer image for gcr.io/distroless/base-debian11:latest
 e03afa0858f2
Step 7/10 : WORKDIR /
 Running in 612720379db3
Removing intermediate container 612720379db3
 a99eee629fff
Step 8/10 : COPY --from=builder /quickstart-go /quickstart-go
 2f87096a1000
Step 9/10 : USER nonroot:nonroot
 Running in 4bf7c47d9c85
Removing intermediate container 4bf7c47d9c85
 2e857568ccd7
Step 10/10 : ENTRYPOINT ["/quickstart-go"]
 Running in 8348fa8162e4
Removing intermediate container 8348fa8162e4
 ca94705aac0a
Successfully built ca94705aac0a
Successfully tagged us-central1-docker.pkg.dev/svo-mvp/hello-repo/helloworld-gke:latest
PUSH
Pushing us-central1-docker.pkg.dev/svo-mvp/hello-repo/helloworld-gke
The push refers to repository [us-central1-docker.pkg.dev/svo-mvp/hello-repo/helloworld-gke]
fdf7dbec2fc2: Preparing
6a1069d9378c: Preparing
1c47a89b8f41: Preparing

7bea6b893187: Pushed
latest: digest: sha256:87d504325a5b176d88e99e562f1e394d309659a04894df33ff88090682d58074 size: 3033
DONE
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
ID: 4cd4f29d-f1f0-49cb-a01a-55d9026684dd
CREATE_TIME: 2023-07-31T20:24:38+00:00
DURATION: 52S
SOURCE: gs://svo-mvp_cloudbuild/source/1690835076.632607-cf8690a62ffa48eba3f560f17aa9ae4f.tgz
IMAGES: us-central1-docker.pkg.dev/svo-mvp/hello-repo/helloworld-gke (+1 more)
STATUS: SUCCESS
ketan_patel@cloudshell:~/helloworld-gke (svo-mvp)$ 




ketan_patel@cloudshell:~/helloworld-gke (svo-mvp)$ gcloud container clusters create-auto helloworld-gke \
  --location us-central1

Creating cluster helloworld-gke in us-central1... Cluster is being health-checked (master is healthy)...done.                            
Created [https://container.googleapis.com/v1/projects/svo-mvp/zones/us-central1/clusters/helloworld-gke].
To inspect the contents of your cluster, go to: https://console.cloud.google.com/kubernetes/workload_/gcloud/us-central1/helloworld-gke?project=svo-mvp
kubeconfig entry generated for helloworld-gke.
NAME: helloworld-gke
LOCATION: us-central1
MASTER_VERSION: 1.27.2-gke.1200
MASTER_IP: 34.29.128.165
MACHINE_TYPE: e2-medium
NODE_VERSION: 1.27.2-gke.1200
NUM_NODES: 3
STATUS: RUNNING
ketan_patel@cloudshell:~/helloworld-gke (svo-mvp)$


ketan_patel@cloudshell:~/helloworld-gke (svo-mvp)$ kubectl config get-clusters

NAME

gke_svo-mvp_us-central1_helloworld-gke

ketan_patel@cloudshell:~/helloworld-gke (svo-mvp)$ kubectl config current-context                                                         

gke_svo-mvp_us-central1_helloworld-gke

ketan_patel@cloudshell:~/helloworld-gke (svo-mvp)$ kubectl get nodes

NAME                                            STATUS   ROLES    AGE   VERSION
gk3-helloworld-gke-default-pool-982b3293-5f23   Ready    <none>   24m   v1.27.2-gke.1200
gk3-helloworld-gke-default-pool-ccaa9cb9-06h2   Ready    <none>   25m   v1.27.2-gke.1200
gk3-helloworld-gke-default-pool-ccaa9cb9-wd6z   Ready    <none>   16m   v1.27.2-gke.1200
ketan_patel@cloudshell:~/helloworld-gke (svo-mvp)$ 


Deploying to GKE

To deploy your app to the GKE cluster you created, you need two Kubernetes objects.


A Deployment to define your app.
A Service to define how to access your app.


Deploy an app

The app has a frontend server that handles the web requests. You define the cluster resources needed to run the frontend in a new file called deployment.yaml. These resources are described as a Deployment. You use Deployments to create and update a ReplicaSet and its associated Pods.

Create the deployment.yaml file in the same directory as your other files and copy the following content.


apiVersion: apps/v1
kind: Deployment
metadata:
  name: helloworld-gke
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hello
  template:
    metadata:
      labels:
        app: hello
    spec:
      containers:
      - name: hello-app
        # Replace $LOCATION with your Artifact Registry location (e.g., us-west1).
        # Replace $GCLOUD_PROJECT with your project ID.
        image: $LOCATION-docker.pkg.dev/$GCLOUD_PROJECT/hello-repo/helloworld-gke:latest
        # This app listens on port 8080 for web traffic by default.
        ports:
        - containerPort: 8080
        env:
          - name: PORT
            value: "8080"
        resources:
          requests:
            memory: "1Gi"
            cpu: "500m"
            ephemeral-storage: "1Gi"
          limits:
            memory: "1Gi"
            cpu: "500m"
            ephemeral-storage: "1Gi"
---



Deploy the resource to the cluster:


ketan_patel@cloudshell:~/helloworld-gke (svo-mvp)$ kubectl apply -f deployment.yaml

deployment.apps/helloworld-gke created

ketan_patel@cloudshell:~/helloworld-gke (svo-mvp)$ kubectl get pods

NAME                              READY   STATUS    RESTARTS   AGE
helloworld-gke-565965576f-pjpnt   1/1     Running   0          85s

ketan_patel@cloudshell:~/helloworld-gke (svo-mvp)$ kubectl get deployment

NAME             READY   UP-TO-DATE   AVAILABLE   AGE
helloworld-gke   1/1     1            1           91s


Deploy a Service

Services provide a single point of access to a set of Pods. While it's possible to access a single Pod, Pods are ephemeral and can only be accessed reliably by using a service address. In your Hello World app, the "hello" Service defines a load balancer to access the hello-app Pods from a single IP address. This service is defined in the service.yaml file.

ketan_patel@cloudshell:~/helloworld-gke (svo-mvp)$cat service.yaml 

# The hello service provides a load-balancing proxy over the hello-app
# pods. By specifying the type as a 'LoadBalancer', Kubernetes Engine will
# create an external HTTP load balancer.
apiVersion: v1
kind: Service
metadata:
  name: hello
spec:
  type: LoadBalancer
  selector:
    app: hello
  ports:
  - port: 80
    targetPort: 8080


The Pods are defined separately from the service that uses the Pods. Kubernetes uses labels to select the pods that a service addresses. With labels, you can have a service that addresses Pods from different replica sets and have multiple services that point to an individual Pod.

Create the Hello World Service:

ketan_patel@cloudshell:~/helloworld-gke (svo-mvp)$ kubectl apply -f service.yaml                                                       

service/hello created

Get the external IP address of the service:


ketan_patel@cloudshell:~/helloworld-gke (svo-mvp)$ kubectl get svc

NAME         TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
hello        LoadBalancer   34.118.231.115   <pending>     80:32643/TCP   10s
kubernetes   ClusterIP      34.118.224.1     <none>        443/TCP        35m


ketan_patel@cloudshell:~/helloworld-gke (svo-mvp)$ curl http://34.135.50.168/

Hello World!
ketan_patel@cloudshell:~/helloworld-gke (svo-mvp)$







GKE - Google Kubernetes Engine









 











ketan_patel@cloudshell:~ (svo-mvp)$ kubectl config get-clusters

NAME
gke_svo-mvp_us-west1_ketan-autopilot-cluster-1

ketan_patel@cloudshell:~ (svo-mvp)$ kubectl config current-context

gke_svo-mvp_us-west1_ketan-autopilot-cluster-1

ketan_patel@cloudshell:~ (svo-mvp)$ kubectl config view

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://34.83.58.26
  name: gke_svo-mvp_us-west1_ketan-autopilot-cluster-1
contexts:
- context:
    cluster: gke_svo-mvp_us-west1_ketan-autopilot-cluster-1
    user: gke_svo-mvp_us-west1_ketan-autopilot-cluster-1
  name: gke_svo-mvp_us-west1_ketan-autopilot-cluster-1
current-context: gke_svo-mvp_us-west1_ketan-autopilot-cluster-1
kind: Config
preferences: {}
users:
- name: gke_svo-mvp_us-west1_ketan-autopilot-cluster-1
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      args: null
      command: gke-gcloud-auth-plugin
      env: null
      installHint: Install gke-gcloud-auth-plugin for use with kubectl by following
        https://cloud.google.com/blog/products/containers-kubernetes/kubectl-auth-changes-in-gke
      interactiveMode: IfAvailable
      provideClusterInfo: true
ketan_patel@cloudshell:~ (svo-mvp)$



ketan_patel@cloudshell:~ (svo-mvp)$ kubectl get namespaces
NAME                       STATUS   AGE
default                    Active   29m
gke-gmp-system             Active   28m
gke-managed-filestorecsi   Active   28m
gmp-public                 Active   28m
kube-node-lease            Active   30m
kube-public                Active   30m
kube-system                Active   30m
ketan_patel@cloudshell:~ (svo-mvp)$ 

ketan_patel@cloudshell:~ (svo-mvp)$ kubectl get deployments -A
NAMESPACE        NAME                                      READY   UP-TO-DATE   AVAILABLE   AGE
gke-gmp-system   gmp-operator                              1/1     1            1           28m
gke-gmp-system   rule-evaluator                            1/1     1            1           28m
kube-system      antrea-controller                         0/0     0            0           28m
kube-system      antrea-controller-horizontal-autoscaler   1/1     1            1           28m
kube-system      egress-nat-controller                     1/1     1            1           28m
kube-system      event-exporter-gke                        1/1     1            1           28m
kube-system      konnectivity-agent                        2/2     2            2           28m
kube-system      konnectivity-agent-autoscaler             1/1     1            1           28m













ketan_patel@cloudshell:~ (svo-mvp)$ kubectl get namespaces

NAME                       STATUS   AGE
default                    Active   29m
gke-gmp-system             Active   28m
gke-managed-filestorecsi   Active   28m
gmp-public                 Active   28m
kube-node-lease            Active   30m
kube-public                Active   30m
kube-system                Active   30m


ketan_patel@cloudshell:~ (svo-mvp)$ kubectl get deployments -A

NAMESPACE        NAME                                      READY   UP-TO-DATE   AVAILABLE   AGE
gke-gmp-system   gmp-operator                              1/1     1            1           28m
gke-gmp-system   rule-evaluator                            1/1     1            1           28m
kube-system      antrea-controller                         0/0     0            0           28m
kube-system      antrea-controller-horizontal-autoscaler   1/1     1            1           28m
kube-system      egress-nat-controller                     1/1     1            1           28m
kube-system      event-exporter-gke                        1/1     1            1           28m
kube-system      konnectivity-agent                        2/2     2            2           28m
kube-system      konnectivity-agent-autoscaler             1/1     1            1           28m
kube-system      kube-dns                                  2/2     2            2           29m
kube-system      kube-dns-autoscaler                       1/1     1            1           29m
kube-system      l7-default-backend                        1/1     1            1           28m
kube-system      metrics-server-v0.5.2                     1/1     1            1           28m

ketan_patel@cloudshell:~ (svo-mvp)$ kubectl get services -A

NAMESPACE        NAME                   TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)            AGE
default          kubernetes             ClusterIP   10.27.0.1     <none>        443/TCP            30m
gke-gmp-system   alertmanager           ClusterIP   None          <none>        9093/TCP           28m
gke-gmp-system   gmp-operator           ClusterIP   10.27.2.37    <none>        8443/TCP,443/TCP   28m
kube-system      antrea                 ClusterIP   10.27.2.132   <none>        443/TCP            28m
kube-system      default-http-backend   NodePort    10.27.0.90    <none>        80:30790/TCP       28m
kube-system      kube-dns               ClusterIP   10.27.0.10    <none>        53/UDP,53/TCP      29m
kube-system      kube-dns-upstream      ClusterIP   10.27.3.41    <none>        53/UDP,53/TCP      28m
kube-system      metrics-server         ClusterIP   10.27.2.12    <none>        443/TCP            28m
ketan_patel@cloudshell:~ (svo-mvp)$ kubectl get

ketan_patel@cloudshell:~ (svo-mvp)$ kubectl get pods -A

NAMESPACE        NAME                                                       READY   STATUS    RESTARTS      AGE
gke-gmp-system   alertmanager-0                                             2/2     Running   0             29m
gke-gmp-system   collector-2l5zv                                            2/2     Running   2 (25m ago)   25m
gke-gmp-system   collector-95kn2                                            2/2     Running   1 (25m ago)   25m
gke-gmp-system   gmp-operator-6b4b9c87d5-flf4h                              1/1     Running   0             29m
gke-gmp-system   rule-evaluator-574fb8bc8f-bstrs                            2/2     Running   3 (25m ago)   26m
kube-system      anetd-llfjx                                                1/1     Running   0             28m
kube-system      anetd-m4956                                                1/1     Running   0             28m
kube-system      antrea-controller-horizontal-autoscaler-548545d58c-p48cs   1/1     Running   0             28m
kube-system      egress-nat-controller-6c9d87ff55-zh92h                     1/1     Running   0             28m
kube-system      event-exporter-gke-d4b7ff94-xqlhz                          2/2     Running   0             29m
kube-system      filestore-node-75jvv                                       3/3     Running   0             28m
kube-system      filestore-node-jsscp                                       3/3     Running   0             28m
kube-system      fluentbit-gke-small-gcjnm                                  2/2     Running   0             28m
kube-system      fluentbit-gke-small-xxlw5                                  2/2     Running   0             28m
kube-system      gcsfusecsi-node-97tfg                                      2/2     Running   0             28m
kube-system      gcsfusecsi-node-fvldm                                      2/2     Running   0             28m
kube-system      gke-metadata-server-rpjhk                                  1/1     Running   0             28m
kube-system      gke-metadata-server-t5fc4                                  1/1     Running   0             28m
kube-system      gke-metrics-agent-hm4sq                                    2/2     Running   0             28m
kube-system      gke-metrics-agent-zf9zb                                    2/2     Running   0             28m
kube-system      image-package-extractor-jp9vs                              1/1     Running   0             28m
kube-system      image-package-extractor-lfncw                              1/1     Running   0             28m
kube-system      ip-masq-agent-hqffc                                        1/1     Running   0             28m
kube-system      ip-masq-agent-qczgl                                        1/1     Running   0             28m
kube-system      konnectivity-agent-7f6464687f-rt5st                        1/1     Running   0             29m
kube-system      konnectivity-agent-7f6464687f-v8zg9                        1/1     Running   0             26m
kube-system      konnectivity-agent-autoscaler-864fff96c4-265fs             1/1     Running   0             29m
kube-system      kube-dns-autoscaler-758c4689b9-q45pp                       1/1     Running   0             29m
kube-system      kube-dns-fc686db9b-dgf6m                                   4/4     Running   0             29m
kube-system      kube-dns-fc686db9b-qxz6s                                   4/4     Running   0             26m
kube-system      l7-default-backend-9db4bd868-j5xvl                         1/1     Running   0             29m
kube-system      metrics-server-v0.5.2-66bbcdbffc-5hmpt                     2/2     Running   0             25m
kube-system      netd-rql94                                                 1/1     Running   0             28m
kube-system      netd-zcx5s                                                 1/1     Running   0             28m
kube-system      node-local-dns-62s7f                                       1/1     Running   0             28m
kube-system      node-local-dns-d2fsc                                       1/1     Running   0             28m
kube-system      pdcsi-node-jsj7z                                           2/2     Running   0             28m
kube-system      pdcsi-node-t6z8x                                           2/2     Running   0             28m
ketan_patel@cloudshell:~ (svo-mvp)$ 

create a Data Fusion instance and deploy a sample pipeline

Create a Data Fusion instance and deploy a sample pipeline that reads an input file from Cloud Storage, transforms and filters the data to o...