aaaa

Create a continuous export pipeline to Pub/Sub

Security Command Center can export security findings to external resources using several methods, including the following:

Continuous exports to a BigQuery dataset.

Continuous exports to Pub/Sub.

One-time exports to CSV files.

One-time exports to Cloud Storage buckets as JSON files.

In this task, you explore how to configure continuous exports of findings to Pub/Sub.

Note: Continuous exports of findings work only for newly created findings.

Continuous exports to Pub/Sub are typically used for forwarding findings to external security management systems such as Splunk or QRadar.

For the purposes of this lab, you export your findings to a Pub/Sub topic and then simulate an application by fetching the messages from a Pub/Sub subscription.

Create new findings to export to Pub/Sub

In this section, you create new findings and check how they are exported to Pub/Sub.

gcloud compute instances create instance-1 --zone=us-west1-c \

--machine-type e2-micro \

--scopes=https://www.googleapis.com/auth/cloud-platform

student_02_93549b8fcf3c@cloudshell:~ (qwiklabs-gcp-04-540e3460bb8f)$ gcloud compute instances create instance-1 --zone=us-west1-c \

--machine-type e2-micro \

--scopes=https://www.googleapis.com/auth/cloud-platform

Created [https://www.googleapis.com/compute/v1/projects/qwiklabs-gcp-04-540e3460bb8f/zones/us-west1-c/instances/instance-1].

NAME: instance-1

ZONE: us-west1-c

MACHINE_TYPE: e2-micro

PREEMPTIBLE: 

INTERNAL_IP: 10.138.0.2

EXTERNAL_IP: 8.229.24.2

STATUS: RUNNING

student_02_93549b8fcf3c@cloudshell:~ (qwiklabs-gcp-04-540e3460bb8f)$ 

Task 2. Export and analyze SCC findings with BigQuery

SCC findings can also be exported to a BigQuery dataset. This might be useful for building analytical dashboards that you can use to check what type of findings appear in your organization most often.

PROJECT_ID=$(gcloud config get project)
bq --location=us-west1 --apilog=/dev/null mk --dataset \
$PROJECT_ID:continuous_export_dataset


student_02_93549b8fcf3c@cloudshell:~ (qwiklabs-gcp-04-540e3460bb8f)$ PROJECT_ID=$(gcloud config get project)
bq --location=us-west1 --apilog=/dev/null mk --dataset \
$PROJECT_ID:continuous_export_dataset
Your active configuration is: [cloudshell-5108]
Dataset 'qwiklabs-gcp-04-540e3460bb8f:continuous_export_dataset' successfully created.
student_02_93549b8fcf3c@cloudshell:~ (qwiklabs-gcp-04-540e3460bb8f)$ 


You have not used an SCC command line interface in this project yet, so you need to enable the SCC service. Run the following command to enable the service in the current project:
gcloud services enable securitycenter.googleapis.com
Copied!

Now create a new export by entering this command:
gcloud scc bqexports create scc-bq-cont-export --dataset=projects/qwiklabs-gcp-04-540e3460bb8f/datasets/continuous_export_dataset --project=qwiklabs-gcp-04-540e3460bb8f


student_02_93549b8fcf3c@cloudshell:~ (qwiklabs-gcp-04-540e3460bb8f)$ gcloud services enable securitycenter.googleapis.com
Operation "operations/acat.p2-189803660465-c1b42e89-d9db-434d-bbf6-38e5f2dc2650" finished successfully.


student_02_93549b8fcf3c@cloudshell:~ (qwiklabs-gcp-04-540e3460bb8f)$ gcloud scc bqexports create scc-bq-cont-export --dataset=projects/qwiklabs-gcp-04-540e3460bb8f/datasets/continuous_export_dataset --project=qwiklabs-gcp-04-540e3460bb8f
Created.
dataset: projects/qwiklabs-gcp-04-540e3460bb8f/datasets/continuous_export_dataset
mostRecentEditor: student-02-93549b8fcf3c@qwiklabs.net
name: projects/189803660465/bigQueryExports/scc-bq-cont-export
principal: service-org-616463121992@gcp-sa-scc-notification.iam.gserviceaccount.com
updateTime: '2026-04-21T20:57:48.665698Z'

student_02_93549b8fcf3c@cloudshell:~ (qwiklabs-gcp-04-540e3460bb8f)$ 


Run the following commands to create three new service accounts without any IAM permissions and create three user-managed service account keys for them.


student_02_93549b8fcf3c@cloudshell:~ (qwiklabs-gcp-04-540e3460bb8f)$ for i in {0..2}; do
gcloud iam service-accounts create sccp-test-sa-$i;
gcloud iam service-accounts keys create /tmp/sa-key-$i.json \
--iam-account=sccp-test-sa-$i@qwiklabs-gcp-04-540e3460bb8f.iam.gserviceaccount.com;
done


Created service account [sccp-test-sa-0].
ERROR: (gcloud.iam.service-accounts.keys.create) NOT_FOUND: Service account projects/-/serviceAccounts/sccp-test-sa-0@qwiklabs-gcp-04-540e3460bb8f.iam.gserviceaccount.com does not exist. This command is authenticated as student-02-93549b8fcf3c@qwiklabs.net which is the active account specified by the [core/account] property.
Created service account [sccp-test-sa-1].
created key [dee7fd4338dda5c54bb83483bc7b66d02e081c54] of type [json] as [/tmp/sa-key-1.json] for [sccp-test-sa-1@qwiklabs-gcp-04-540e3460bb8f.iam.gserviceaccount.com]
Created service account [sccp-test-sa-2].
created key [97adcb1c5ab74f5767050e92ec5921857e5c7c5c] of type [json] as [/tmp/sa-key-2.json] for [sccp-test-sa-2@qwiklabs-gcp-04-540e3460bb8f.iam.gserviceaccount.com]
student_02_93549b8fcf3c@cloudshell:~ (qwiklabs-gcp-04-540e3460bb8f)$ 



Once new findings are created in SCC, they are exported to BigQuery. For storing them, the export pipeline creates a new table called findings.

Run the following command to fetch information from BigQuery about newly created findings:
bq query --apilog=/dev/null --use_legacy_sql=false  \
"SELECT finding_id,event_time,finding.category FROM continuous_export_dataset.findings"



Export findings to a Cloud Storage bucket and create a BigQuery table
Security Command Center is typically enabled in pre-existing and mature Google Cloud infrastructures. As soon as the SCC is enabled, it starts scanning existing vulnerabilities and eventually might report thousands of findings on existing infrastructure.

The SCC interface might not provide the best way to sort and filter such findings, so exporting these findings to a BigQuery database is a common practice for running analytics against findings.

Direct exporting of findings to BigQuery is not supported yet. Instead, you can use a Google Cloud Storage bucket as an interim storage solution.

Create a Cloud Storage bucket



xport existing findings as JSONL data
In this section, you export your findings for use in a BigQuery database.

In the Cloud console, on the Navigation menu (Navigation menu icon), click Security > Findings.

Click the Export button.