Showing posts with label Grafana. Show all posts
Showing posts with label Grafana. Show all posts

Migrate Existing Prometheus Monitoring Workloads to Google Cloud

Overview
In this lab, you will explore how to use Managed Service for Prometheus in a self-deployed data collection mode. You can also utilize managed data collection as well.

With self-deployed data collections, you manage your Prometheus installation as usual. The only difference from upstream Prometheus is that you run the Managed Service for Prometheus drop-in replacement binary instead of the upstream Prometheus binary.

You can find more information on considerations to make when choosing a managed vs. self-managed data collection at the following documentation link: Data collection with Managed Service for Prometheus.


Deploy the Managed Service for Prometheus
Create a self managed data collection for scraping metrics
Understand considerations to make when using managed vs. self-managed data collections
Utilize Grafana to query Prometheus metrics data




ketan_patel@cloudshell:~ (new-user-learning)$ kubectl -n gmp-test apply -f https://raw.githubusercontent.com/GoogleCloudPlatform/prometheus-engine/v0.4.3-gke.0/examples/example-app.yaml

deployment.apps/prom-example created

ketan_patel@cloudshell:~ (new-user-learning)$ kubectl -n gmp-test apply -f https://raw.githubusercontent.com/GoogleCloudPlatform/prometheus-engine/v0.4.3-gke.0/examples/prometheus.yaml

clusterrole.rbac.authorization.k8s.io/gmp-test:prometheus-test created
clusterrolebinding.rbac.authorization.k8s.io/gmp-test:prometheus-test created
service/prometheus-test created
statefulset.apps/prometheus-test created
configmap/prometheus-test created

ketan_patel@cloudshell:~ (new-user-learning)$ kubectl -n gmp-test get pod

NAME                              READY   STATUS    RESTARTS      AGE
helloworld-gke-5f574446d7-97gx7   1/1     Running   0             80m
prom-example-7987cfb88f-6wnq2     1/1     Running   0             54s
prom-example-7987cfb88f-bv5f5     1/1     Running   0             54s
prom-example-7987cfb88f-thsqv     1/1     Running   0             54s
prometheus-test-0                 2/2     Running   1 (11s ago)   20s

ketan_patel@cloudshell:~ (new-user-learning)$ export PROJECT_ID=$(gcloud config get-value project)

Your active configuration is: [cloudshell-533]

ketan_patel@cloudshell:~ (new-user-learning)$ curl https://raw.githubusercontent.com/GoogleCloudPlatform/prometheus-engine/v0.4.3-gke.0/examples/frontend.yaml |

sed "s/\$PROJECT_ID/$PROJECT_ID/" | kubectl apply -n gmp-test -f -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1514  100  1514    0     0   8096      0 --:--:-- --:--:-- --:--:--  8096
deployment.apps/frontend created
service/frontend created

ketan_patel@cloudshell:~ (new-user-learning)$ kubectl -n gmp-test port-forward svc/frontend 9090

Forwarding from 127.0.0.1:9090 -> 9090
Handling connection for 9090
Handling connection for 9090






ketan_patel@cloudshell:~/kube-prometheus (new-user-learning)$ kubectl -n gmp-test apply -f https://raw.githubusercontent.com/GoogleCloudPlatform/prometheus-engine/v0.4.3-gke.0/examples/grafana.yaml
deployment.apps/grafana created
service/grafana created


ketan_patel@cloudshell:~/kube-prometheus (new-user-learning)$ kubectl -n gmp-test port-forward svc/grafana 3001:3000
error: unable to forward port because pod is not running. Current status=Pending


ketan_patel@cloudshell:~/kube-prometheus (new-user-learning)$ kubectl get pods -n gmp-test
NAME                              READY   STATUS    RESTARTS        AGE
frontend-694bd6ff76-2wcr5         1/1     Running   0               8m17s
frontend-694bd6ff76-qp4bs         1/1     Running   0               8m17s
grafana-9fdc4b86b-22wwr           1/1     Running   0               31s
helloworld-gke-5f574446d7-97gx7   1/1     Running   0               90m
prom-example-7987cfb88f-6wnq2     1/1     Running   0               10m
prom-example-7987cfb88f-bv5f5     1/1     Running   0               10m
prom-example-7987cfb88f-thsqv     1/1     Running   0               10m
prometheus-test-0                 2/2     Running   1 (9m42s ago)   9m51s
 
ketan_patel@cloudshell:~/kube-prometheus (new-user-learning)$ kubectl -n gmp-test port-forward svc/grafana 3001:3000
Forwarding from 127.0.0.1:3001 -> 3000
Handling connection for 3001
Handling connection for 3001






create a Data Fusion instance and deploy a sample pipeline

Create a Data Fusion instance and deploy a sample pipeline that reads an input file from Cloud Storage, transforms and filters the data to o...