Service accounts are a special type of Google account that grant permissions to virtual machines instead of end users. Service accounts are primarily used to ensure safe, managed connections to APIs and Google Cloud services. Granting access to trusted connections and rejecting malicious ones is a must-have security feature for any Google Cloud project. In this lab, you get hands-on practice with the ins and outs of service accounts.
Create and manage service accounts.
Create a virtual machine and associate it with a service account.
Use client libraries to access BigQuery from a service account.
Run a query on a BigQuery public dataset from a Compute Engine instance.
What are service accounts?
A service account is a special Google account that belongs to your application or a virtual machine (VM) instead of an individual end user. Your application uses the service account to call the Google API of a service, so that the users aren't directly involved.
For example, a Compute Engine VM may run as a service account, and that account can be given permissions to access the resources it needs. This way the service account is the identity of the service, and the service account's permissions control which resources the service can access.
A service account is identified by its email address, which is unique to the account.
Types of service accounts
User-managed service accounts
When you create a new Cloud project using Google Cloud console and if Compute Engine API is enabled for your project, a Compute Engine Service account is created for you by default. It is identifiable using the email:
PROJECT_NUMBER-compute@developer.gserviceaccount.com
If your project contains an App Engine application, the default App Engine service account is created in your project by default. It is identifiable using the email:
PROJECT_ID@appspot.gserviceaccount.com
Google-managed service accounts
In addition to the user-managed service accounts, you might see some additional service accounts in your project’s IAM policy or in the console. These service accounts are created and owned by Google. These accounts represent different Google services and each account is automatically granted IAM roles to access your Google Cloud project.
Google APIs service account
An example of a Google-managed service account is a Google API service account identifiable using the email:
PROJECT_NUMBER@cloudservices.gserviceaccount.com
This service account is designed specifically to run internal Google processes on your behalf and is not listed in the Service Accounts section of the console. By default, the account is automatically granted the project editor role on the project and is listed in the IAM section of the console. This service account is deleted only when the project is deleted.
Note: Google services rely on the account having access to your project, so you should not remove or change the service account’s role on your project.
Task 1. Creating and managing service accounts
When you create a new Cloud project, Google Cloud automatically creates one Compute Engine service account and one App Engine service account under that project. You can create up to 98 additional service accounts to your project to control access to your resources.
Creating a service account
Creating a service account is similar to adding a member to your project, but the service account belongs to your applications rather than an individual end user.
student_04_717d1aa28d01@cloudshell:~ (qwiklabs-gcp-03-f3d72bb5968f)$ history
1 gcloud compute ssh source-instance --zone=$ZONE
2 gcloud auth list
3 gauth config list
4 gcloud config list
5 PROJECT_NUMBER-compute@developer.gserviceaccount.com
6
7
8 gcloud config set compute/region us-west4
9 gcloud config list project
10 gcloud iam service-accounts create my-sa-123 --display-name "my service account"
11 gcloud projects add-iam-policy-binding $DEVSHELL_PROJECT_ID --member serviceAccount:my-sa-123@$DEVSHELL_PROJECT_ID.iam.gserviceaccount.com --role roles/editor
Linux bigquery-instance 5.10.0-23-cloud-amd64 #1 SMP Debian 5.10.179-1 (2023-05-12) x86_64
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Creating directory '/home/student-04-717d1aa28d01'.
student-04-717d1aa28d01@bigquery-instance:~$ sudo apt-get update
Hit:1 https://deb.debian.org/debian bullseye InRelease
Get:2 https://deb.debian.org/debian-security bullseye-security InRelease [48.4 kB]
Get:3 https://packages.cloud.google.com/apt google-compute-engine-bullseye-stable InRelease [5146 B]
Get:4 https://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]
Get:5 https://deb.debian.org/debian bullseye-backports InRelease [49.0 kB]
Get:6 https://packages.cloud.google.com/apt cloud-sdk-bullseye InRelease [6406 B]
Get:7 https://deb.debian.org/debian-security bullseye-security/main Sources [204 kB]
Get:8 https://deb.debian.org/debian-security bullseye-security/main amd64 Packages [252 kB]
Get:9 https://deb.debian.org/debian-security bullseye-security/main Translation-en [164 kB]
Get:10 https://packages.cloud.google.com/apt google-compute-engine-bullseye-stable/main amd64 Packages [1942 B]
Get:11 https://deb.debian.org/debian bullseye-updates/main Sources.diff/Index [19.6 kB]
Get:12 https://deb.debian.org/debian bullseye-updates/main amd64 Packages.diff/Index [19.6 kB]
Get:13 https://deb.debian.org/debian bullseye-updates/main Translation-en.diff/Index [8361 B]
Get:14 https://deb.debian.org/debian bullseye-updates/main Sources T-2023-07-31-2005.11-F-2023-07-31-2005.11.pdiff [1673 B]
Get:15 https://deb.debian.org/debian bullseye-updates/main amd64 Packages T-2023-07-31-2005.11-F-2023-07-31-2005.11.pdiff [3382 B]
Get:14 https://deb.debian.org/debian bullseye-updates/main Sources T-2023-07-31-2005.11-F-2023-07-31-2005.11.pdiff [1673 B]
Get:15 https://deb.debian.org/debian bullseye-updates/main amd64 Packages T-2023-07-31-2005.11-F-2023-07-31-2005.11.pdiff [3382 B]
Get:16 https://deb.debian.org/debian bullseye-updates/main Translation-en T-2023-07-31-2005.11-F-2023-07-31-2005.11.pdiff [2273 B]
Get:16 https://deb.debian.org/debian bullseye-updates/main Translation-en T-2023-07-31-2005.11-F-2023-07-31-2005.11.pdiff [2273 B]
Get:17 https://deb.debian.org/debian bullseye-backports/main Sources.diff/Index [63.3 kB]
Get:18 https://deb.debian.org/debian bullseye-backports/main amd64 Packages.diff/Index [63.3 kB]
Get:19 https://deb.debian.org/debian bullseye-backports/main Translation-en.diff/Index [63.3 kB]
Get:20 https://deb.debian.org/debian bullseye-backports/main Sources T-2023-07-18-0816.49-F-2023-07-18-0209.04.pdiff [28.3 kB]
Get:20 https://deb.debian.org/debian bullseye-backports/main Sources T-2023-07-18-0816.49-F-2023-07-18-0209.04.pdiff [28.3 kB]
Get:21 https://deb.debian.org/debian bullseye-backports/main amd64 Packages T-2023-07-18-1410.09-F-2023-07-18-0209.04.pdiff [20.1 kB]
Get:21 https://deb.debian.org/debian bullseye-backports/main amd64 Packages T-2023-07-18-1410.09-F-2023-07-18-0209.04.pdiff [20.1 kB]
Get:22 https://deb.debian.org/debian bullseye-backports/main Translation-en T-2023-07-18-2011.14-F-2023-07-18-0209.04.pdiff [5503 B]
Get:23 https://packages.cloud.google.com/apt cloud-sdk-bullseye/main amd64 Packages [339 kB]
Get:22 https://deb.debian.org/debian bullseye-backports/main Translation-en T-2023-07-18-2011.14-F-2023-07-18-0209.04.pdiff [5503 B]
Fetched 1413 kB in 1s (1222 kB/s)
Reading package lists... Done
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$ sudo apt-get install -y git python3-pip
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
binutils binutils-common binutils-x86-64-linux-gnu build-essential bzip2 cpp cpp-10 dpkg-dev fakeroot
fontconfig-config fonts-dejavu-core g++ g++-10 gcc gcc-10 git-man javascript-common libalgorithm-diff-perl
libalgorithm-diff-xs-perl libalgorithm-merge-perl libasan6 libatomic1 libbinutils libc-dev-bin
libc-devtools libc6-dev libcc1-0 libcrypt-dev libctf-nobfd0 libctf0 libdeflate0 libdpkg-perl liberror-perl
libexpat1-dev libfakeroot libfile-fcntllock-perl libfontconfig1 libgcc-10-dev libgd3 libgdbm-compat4
libgomp1 libisl23 libitm1 libjbig0 libjpeg62-turbo libjs-jquery libjs-sphinxdoc libjs-underscore
liblocale-gettext-perl liblsan0 libmpc3 libmpfr6 libnsl-dev libperl5.32 libpython3-dev libpython3.9-dev
libquadmath0 libstdc++-10-dev libtiff5 libtirpc-dev libtsan0 libubsan1 libwebp6 libx11-6 libx11-data
libxau6 libxcb1 libxdmcp6 libxpm4 linux-libc-dev make manpages-dev patch perl perl-modules-5.32
python-pip-whl python3-dev python3-distutils python3-lib2to3 python3-setuptools python3-wheel python3.9-dev
zlib1g-dev
Suggested packages:
binutils-doc bzip2-doc cpp-doc gcc-10-locales debian-keyring g++-multilib g++-10-multilib gcc-10-doc
gcc-multilib autoconf automake libtool flex bison gdb gcc-doc gcc-10-multilib git-daemon-run
| git-daemon-sysvinit git-doc git-el git-email git-gui gitk gitweb git-cvs git-mediawiki git-svn apache2
| lighttpd | httpd glibc-doc bzr libgd-tools libstdc++-10-doc make-doc ed diffutils-doc perl-doc
libterm-readline-gnu-perl | libterm-readline-perl-perl libtap-harness-archive-perl python-setuptools-doc
The following NEW packages will be installed:
binutils binutils-common binutils-x86-64-linux-gnu build-essential bzip2 cpp cpp-10 dpkg-dev fakeroot
fontconfig-config fonts-dejavu-core g++ g++-10 gcc gcc-10 git git-man javascript-common
libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl libasan6 libatomic1 libbinutils
libc-dev-bin libc-devtools libc6-dev libcc1-0 libcrypt-dev libctf-nobfd0 libctf0 libdeflate0 libdpkg-perl
liberror-perl libexpat1-dev libfakeroot libfile-fcntllock-perl libfontconfig1 libgcc-10-dev libgd3
libgdbm-compat4 libgomp1 libisl23 libitm1 libjbig0 libjpeg62-turbo libjs-jquery libjs-sphinxdoc
libjs-underscore liblocale-gettext-perl liblsan0 libmpc3 libmpfr6 libnsl-dev libperl5.32 libpython3-dev
libpython3.9-dev libquadmath0 libstdc++-10-dev libtiff5 libtirpc-dev libtsan0 libubsan1 libwebp6 libx11-6
libx11-data libxau6 libxcb1 libxdmcp6 libxpm4 linux-libc-dev make manpages-dev patch perl perl-modules-5.32
python-pip-whl python3-dev python3-distutils python3-lib2to3 python3-pip python3-setuptools python3-wheel
python3.9-dev zlib1g-dev
0 upgraded, 85 newly installed, 0 to remove and 11 not upgraded.
Need to get 91.4 MB of archives.
After this operation, 358 MB of additional disk space will be used.
Get:1 https://deb.debian.org/debian bullseye/main amd64 perl-modules-5.32 all 5.32.1-4+deb11u2 [2823 kB]
Get:2 https://deb.debian.org/debian bullseye/main amd64 libgdbm-compat4 amd64 1.19-2 [44.7 kB]
Get:3 https://deb.debian.org/debian bullseye/main amd64 libperl5.32 amd64 5.32.1-4+deb11u2 [4106 kB]
Get:4 https://deb.debian.org/debian bullseye/main amd64 perl amd64 5.32.1-4+deb11u2 [293 kB]
Get:5 https://deb.debian.org/debian bullseye/main amd64 liblocale-gettext-perl amd64 1.07-4+b1 [19.0 kB]
Get:6 https://deb.debian.org/debian bullseye/main amd64 bzip2 amd64 1.0.8-4 [49.3 kB]
Get:7 https://deb.debian.org/debian bullseye/main amd64 binutils-common amd64 2.35.2-2 [2220 kB]
Get:8 https://deb.debian.org/debian bullseye/main amd64 libbinutils amd64 2.35.2-2 [570 kB]
Get:9 https://deb.debian.org/debian bullseye/main amd64 libctf-nobfd0 amd64 2.35.2-2 [110 kB]
Get:10 https://deb.debian.org/debian bullseye/main amd64 libctf0 amd64 2.35.2-2 [53.2 kB]
Get:11 https://deb.debian.org/debian bullseye/main amd64 binutils-x86-64-linux-gnu amd64 2.35.2-2 [1809 kB]
Get:12 https://deb.debian.org/debian bullseye/main amd64 binutils amd64 2.35.2-2 [61.2 kB]
Get:13 https://deb.debian.org/debian bullseye/main amd64 libc-dev-bin amd64 2.31-13+deb11u6 [276 kB]
Get:14 https://deb.debian.org/debian-security bullseye-security/main amd64 linux-libc-dev amd64 5.10.179-3 [1619 kB]
Get:15 https://deb.debian.org/debian bullseye/main amd64 libcrypt-dev amd64 1:4.4.18-4 [104 kB]
Get:16 https://deb.debian.org/debian bullseye/main amd64 libtirpc-dev amd64 1.3.1-1+deb11u1 [191 kB]
Get:17 https://deb.debian.org/debian bullseye/main amd64 libnsl-dev amd64 1.3.0-2 [66.4 kB]
Get:18 https://deb.debian.org/debian bullseye/main amd64 libc6-dev amd64 2.31-13+deb11u6 [2360 kB]
Get:19 https://deb.debian.org/debian bullseye/main amd64 libisl23 amd64 0.23-1 [676 kB]
Get:20 https://deb.debian.org/debian bullseye/main amd64 libmpfr6 amd64 4.1.0-3 [2012 kB]
Get:21 https://deb.debian.org/debian bullseye/main amd64 libmpc3 amd64 1.2.0-1 [45.0 kB]
Get:22 https://deb.debian.org/debian bullseye/main amd64 cpp-10 amd64 10.2.1-6 [8528 kB]
Get:23 https://deb.debian.org/debian bullseye/main amd64 cpp amd64 4:10.2.1-1 [19.7 kB]
Get:24 https://deb.debian.org/debian bullseye/main amd64 libcc1-0 amd64 10.2.1-6 [47.0 kB]
Get:25 https://deb.debian.org/debian bullseye/main amd64 libgomp1 amd64 10.2.1-6 [99.9 kB]
Get:26 https://deb.debian.org/debian bullseye/main amd64 libitm1 amd64 10.2.1-6 [25.8 kB]
Get:27 https://deb.debian.org/debian bullseye/main amd64 libatomic1 amd64 10.2.1-6 [9008 B]
Get:28 https://deb.debian.org/debian bullseye/main amd64 libasan6 amd64 10.2.1-6 [2065 kB]
Get:29 https://deb.debian.org/debian bullseye/main amd64 liblsan0 amd64 10.2.1-6 [828 kB]
Get:30 https://deb.debian.org/debian bullseye/main amd64 libtsan0 amd64 10.2.1-6 [2000 kB]
Get:31 https://deb.debian.org/debian bullseye/main amd64 libubsan1 amd64 10.2.1-6 [777 kB]
Get:32 https://deb.debian.org/debian bullseye/main amd64 libquadmath0 amd64 10.2.1-6 [145 kB]
Get:33 https://deb.debian.org/debian bullseye/main amd64 libgcc-10-dev amd64 10.2.1-6 [2328 kB]
Get:34 https://deb.debian.org/debian bullseye/main amd64 gcc-10 amd64 10.2.1-6 [17.0 MB]
Get:35 https://deb.debian.org/debian bullseye/main amd64 gcc amd64 4:10.2.1-1 [5192 B]
Get:36 https://deb.debian.org/debian bullseye/main amd64 libstdc++-10-dev amd64 10.2.1-6 [1741 kB]
Get:37 https://deb.debian.org/debian bullseye/main amd64 g++-10 amd64 10.2.1-6 [9380 kB]
Get:38 https://deb.debian.org/debian bullseye/main amd64 g++ amd64 4:10.2.1-1 [1644 B]
Get:39 https://deb.debian.org/debian bullseye/main amd64 make amd64 4.3-4.1 [396 kB]
Get:40 https://deb.debian.org/debian bullseye/main amd64 libdpkg-perl all 1.20.12 [1551 kB]
Get:41 https://deb.debian.org/debian bullseye/main amd64 patch amd64 2.7.6-7 [128 kB]
Get:42 https://deb.debian.org/debian bullseye/main amd64 dpkg-dev all 1.20.12 [2312 kB]
Get:43 https://deb.debian.org/debian bullseye/main amd64 build-essential amd64 12.9 [7704 B]
Get:44 https://deb.debian.org/debian bullseye/main amd64 libfakeroot amd64 1.25.3-1.1 [47.0 kB]
Get:45 https://deb.debian.org/debian bullseye/main amd64 fakeroot amd64 1.25.3-1.1 [87.0 kB]
Get:46 https://deb.debian.org/debian bullseye/main amd64 fonts-dejavu-core all 2.37-2 [1069 kB]
Get:47 https://deb.debian.org/debian bullseye/main amd64 fontconfig-config all 2.13.1-4.2 [281 kB]
Get:48 https://deb.debian.org/debian bullseye/main amd64 liberror-perl all 0.17029-1 [31.0 kB]
Get:49 https://deb.debian.org/debian bullseye/main amd64 git-man all 1:2.30.2-1+deb11u2 [1828 kB]
Get:50 https://deb.debian.org/debian bullseye/main amd64 git amd64 1:2.30.2-1+deb11u2 [5518 kB]
Get:51 https://deb.debian.org/debian bullseye/main amd64 javascript-common all 11+nmu1 [6260 B]
Get:52 https://deb.debian.org/debian bullseye/main amd64 libalgorithm-diff-perl all 1.201-1 [43.3 kB]
Get:53 https://deb.debian.org/debian bullseye/main amd64 libalgorithm-diff-xs-perl amd64 0.04-6+b1 [12.0 kB]
Get:54 https://deb.debian.org/debian bullseye/main amd64 libalgorithm-merge-perl all 0.08-3 [12.7 kB]
Get:55 https://deb.debian.org/debian bullseye/main amd64 libfontconfig1 amd64 2.13.1-4.2 [347 kB]
Get:56 https://deb.debian.org/debian bullseye/main amd64 libjpeg62-turbo amd64 1:2.0.6-4 [151 kB]
Get:57 https://deb.debian.org/debian bullseye/main amd64 libdeflate0 amd64 1.7-1 [53.1 kB]
Get:58 https://deb.debian.org/debian bullseye/main amd64 libjbig0 amd64 2.1-3.1+b2 [31.0 kB]
Get:59 https://deb.debian.org/debian-security bullseye-security/main amd64 libwebp6 amd64 0.6.1-2.1+deb11u1 [258 kB]
Get:60 https://deb.debian.org/debian bullseye/main amd64 libtiff5 amd64 4.2.0-1+deb11u4 [290 kB]
Get:61 https://deb.debian.org/debian bullseye/main amd64 libxau6 amd64 1:1.0.9-1 [19.7 kB]
Get:62 https://deb.debian.org/debian bullseye/main amd64 libxdmcp6 amd64 1:1.1.2-3 [26.3 kB]
Get:63 https://deb.debian.org/debian bullseye/main amd64 libxcb1 amd64 1.14-3 [140 kB]
Get:64 https://deb.debian.org/debian-security bullseye-security/main amd64 libx11-data all 2:1.7.2-1+deb11u1 [311 kB]
Get:65 https://deb.debian.org/debian-security bullseye-security/main amd64 libx11-6 amd64 2:1.7.2-1+deb11u1 [772 kB]
Get:66 https://deb.debian.org/debian bullseye/main amd64 libxpm4 amd64 1:3.5.12-1.1~deb11u1 [49.6 kB]
Get:67 https://deb.debian.org/debian bullseye/main amd64 libgd3 amd64 2.3.0-2 [137 kB]
Get:68 https://deb.debian.org/debian bullseye/main amd64 libc-devtools amd64 2.31-13+deb11u6 [246 kB]
Get:69 https://deb.debian.org/debian bullseye/main amd64 libexpat1-dev amd64 2.2.10-2+deb11u5 [141 kB]
Get:70 https://deb.debian.org/debian bullseye/main amd64 libfile-fcntllock-perl amd64 0.22-3+b7 [35.5 kB]
Get:71 https://deb.debian.org/debian bullseye/main amd64 libjs-jquery all 3.5.1+dfsg+~3.5.5-7 [315 kB]
Get:72 https://deb.debian.org/debian bullseye/main amd64 libjs-underscore all 1.9.1~dfsg-3 [100 kB]
Get:73 https://deb.debian.org/debian bullseye/main amd64 libjs-sphinxdoc all 3.4.3-2 [127 kB]
Get:74 https://deb.debian.org/debian bullseye/main amd64 libpython3.9-dev amd64 3.9.2-1 [4028 kB]
Get:75 https://deb.debian.org/debian bullseye/main amd64 libpython3-dev amd64 3.9.2-3 [21.7 kB]
Get:76 https://deb.debian.org/debian bullseye/main amd64 manpages-dev all 5.10-1 [2309 kB]
Get:77 https://deb.debian.org/debian bullseye/main amd64 python-pip-whl all 20.3.4-4+deb11u1 [1948 kB]
Get:78 https://deb.debian.org/debian bullseye/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-2+deb11u2 [191 kB]
Get:79 https://deb.debian.org/debian bullseye/main amd64 python3.9-dev amd64 3.9.2-1 [515 kB]
Get:80 https://deb.debian.org/debian bullseye/main amd64 python3-lib2to3 all 3.9.2-1 [77.8 kB]
Get:81 https://deb.debian.org/debian bullseye/main amd64 python3-distutils all 3.9.2-1 [143 kB]
Get:82 https://deb.debian.org/debian bullseye/main amd64 python3-dev amd64 3.9.2-3 [24.8 kB]
Get:83 https://deb.debian.org/debian bullseye/main amd64 python3-setuptools all 52.0.0-4 [366 kB]
Get:84 https://deb.debian.org/debian bullseye/main amd64 python3-wheel all 0.34.2-1 [24.0 kB]
Get:85 https://deb.debian.org/debian bullseye/main amd64 python3-pip all 20.3.4-4+deb11u1 [337 kB]
Fetched 91.4 MB in 1s (97.3 MB/s)
Extracting templates from packages: 100%
Preconfiguring packages ...
Selecting previously unselected package perl-modules-5.32.
(Reading database ... 57457 files and directories currently installed.)
Preparing to unpack .../00-perl-modules-5.32_5.32.1-4+deb11u2_all.deb ...
Unpacking perl-modules-5.32 (5.32.1-4+deb11u2) ...
Selecting previously unselected package libgdbm-compat4:amd64.
Preparing to unpack .../01-libgdbm-compat4_1.19-2_amd64.deb ...
Unpacking libgdbm-compat4:amd64 (1.19-2) ...
Selecting previously unselected package libperl5.32:amd64.
Preparing to unpack .../02-libperl5.32_5.32.1-4+deb11u2_amd64.deb ...
Unpacking libperl5.32:amd64 (5.32.1-4+deb11u2) ...
Selecting previously unselected package perl.
Preparing to unpack .../03-perl_5.32.1-4+deb11u2_amd64.deb ...
Unpacking perl (5.32.1-4+deb11u2) ...
Selecting previously unselected package liblocale-gettext-perl.
Preparing to unpack .../04-liblocale-gettext-perl_1.07-4+b1_amd64.deb ...
Unpacking liblocale-gettext-perl (1.07-4+b1) ...
Selecting previously unselected package bzip2.
Preparing to unpack .../05-bzip2_1.0.8-4_amd64.deb ...
Unpacking bzip2 (1.0.8-4) ...
Selecting previously unselected package binutils-common:amd64.
Preparing to unpack .../06-binutils-common_2.35.2-2_amd64.deb ...
Unpacking binutils-common:amd64 (2.35.2-2) ...
Selecting previously unselected package libbinutils:amd64.
Preparing to unpack .../07-libbinutils_2.35.2-2_amd64.deb ...
Unpacking libbinutils:amd64 (2.35.2-2) ...
Selecting previously unselected package libctf-nobfd0:amd64.
Preparing to unpack .../08-libctf-nobfd0_2.35.2-2_amd64.deb ...
Unpacking libctf-nobfd0:amd64 (2.35.2-2) ...
Selecting previously unselected package libctf0:amd64.
Preparing to unpack .../09-libctf0_2.35.2-2_amd64.deb ...
Unpacking libctf0:amd64 (2.35.2-2) ...
Selecting previously unselected package binutils-x86-64-linux-gnu.
Preparing to unpack .../10-binutils-x86-64-linux-gnu_2.35.2-2_amd64.deb ...
Unpacking binutils-x86-64-linux-gnu (2.35.2-2) ...
Selecting previously unselected package binutils.
Preparing to unpack .../11-binutils_2.35.2-2_amd64.deb ...
Unpacking binutils (2.35.2-2) ...
Selecting previously unselected package libc-dev-bin.
Preparing to unpack .../12-libc-dev-bin_2.31-13+deb11u6_amd64.deb ...
Unpacking libc-dev-bin (2.31-13+deb11u6) ...
Selecting previously unselected package linux-libc-dev:amd64.
Preparing to unpack .../13-linux-libc-dev_5.10.179-3_amd64.deb ...
Unpacking linux-libc-dev:amd64 (5.10.179-3) ...
Selecting previously unselected package libcrypt-dev:amd64.
Preparing to unpack .../14-libcrypt-dev_1%3a4.4.18-4_amd64.deb ...
Unpacking libcrypt-dev:amd64 (1:4.4.18-4) ...
Selecting previously unselected package libtirpc-dev:amd64.
Preparing to unpack .../15-libtirpc-dev_1.3.1-1+deb11u1_amd64.deb ...
Unpacking libtirpc-dev:amd64 (1.3.1-1+deb11u1) ...
Selecting previously unselected package libnsl-dev:amd64.
Preparing to unpack .../16-libnsl-dev_1.3.0-2_amd64.deb ...
Unpacking libnsl-dev:amd64 (1.3.0-2) ...
Selecting previously unselected package libc6-dev:amd64.
Preparing to unpack .../17-libc6-dev_2.31-13+deb11u6_amd64.deb ...
Unpacking libc6-dev:amd64 (2.31-13+deb11u6) ...
Selecting previously unselected package libisl23:amd64.
Preparing to unpack .../18-libisl23_0.23-1_amd64.deb ...
Unpacking libisl23:amd64 (0.23-1) ...
Selecting previously unselected package libmpfr6:amd64.
Preparing to unpack .../19-libmpfr6_4.1.0-3_amd64.deb ...
Unpacking libmpfr6:amd64 (4.1.0-3) ...
Selecting previously unselected package libmpc3:amd64.
Preparing to unpack .../20-libmpc3_1.2.0-1_amd64.deb ...
Unpacking libmpc3:amd64 (1.2.0-1) ...
Selecting previously unselected package cpp-10.
Preparing to unpack .../21-cpp-10_10.2.1-6_amd64.deb ...
Unpacking cpp-10 (10.2.1-6) ...
Selecting previously unselected package cpp.
Preparing to unpack .../22-cpp_4%3a10.2.1-1_amd64.deb ...
Unpacking cpp (4:10.2.1-1) ...
Selecting previously unselected package libcc1-0:amd64.
Preparing to unpack .../23-libcc1-0_10.2.1-6_amd64.deb ...
Unpacking libcc1-0:amd64 (10.2.1-6) ...
Selecting previously unselected package libgomp1:amd64.
Preparing to unpack .../24-libgomp1_10.2.1-6_amd64.deb ...
Unpacking libgomp1:amd64 (10.2.1-6) ...
Selecting previously unselected package libitm1:amd64.
Preparing to unpack .../25-libitm1_10.2.1-6_amd64.deb ...
Unpacking libitm1:amd64 (10.2.1-6) ...
Selecting previously unselected package libatomic1:amd64.
Preparing to unpack .../26-libatomic1_10.2.1-6_amd64.deb ...
Unpacking libatomic1:amd64 (10.2.1-6) ...
Selecting previously unselected package libasan6:amd64.
Preparing to unpack .../27-libasan6_10.2.1-6_amd64.deb ...
Unpacking libasan6:amd64 (10.2.1-6) ...
Selecting previously unselected package liblsan0:amd64.
Preparing to unpack .../28-liblsan0_10.2.1-6_amd64.deb ...
Unpacking liblsan0:amd64 (10.2.1-6) ...
Selecting previously unselected package libtsan0:amd64.
Preparing to unpack .../29-libtsan0_10.2.1-6_amd64.deb ...
Unpacking libtsan0:amd64 (10.2.1-6) ...
Selecting previously unselected package libubsan1:amd64.
Preparing to unpack .../30-libubsan1_10.2.1-6_amd64.deb ...
Unpacking libubsan1:amd64 (10.2.1-6) ...
Selecting previously unselected package libquadmath0:amd64.
Preparing to unpack .../31-libquadmath0_10.2.1-6_amd64.deb ...
Unpacking libquadmath0:amd64 (10.2.1-6) ...
Selecting previously unselected package libgcc-10-dev:amd64.
Preparing to unpack .../32-libgcc-10-dev_10.2.1-6_amd64.deb ...
Unpacking libgcc-10-dev:amd64 (10.2.1-6) ...
Selecting previously unselected package gcc-10.
Preparing to unpack .../33-gcc-10_10.2.1-6_amd64.deb ...
Unpacking gcc-10 (10.2.1-6) ...
Selecting previously unselected package gcc.
Preparing to unpack .../34-gcc_4%3a10.2.1-1_amd64.deb ...
Unpacking gcc (4:10.2.1-1) ...
Selecting previously unselected package libstdc++-10-dev:amd64.
Preparing to unpack .../35-libstdc++-10-dev_10.2.1-6_amd64.deb ...
Unpacking libstdc++-10-dev:amd64 (10.2.1-6) ...
Selecting previously unselected package g++-10.
Preparing to unpack .../36-g++-10_10.2.1-6_amd64.deb ...
Unpacking g++-10 (10.2.1-6) ...
Selecting previously unselected package g++.
Preparing to unpack .../37-g++_4%3a10.2.1-1_amd64.deb ...
Unpacking g++ (4:10.2.1-1) ...
Selecting previously unselected package make.
Preparing to unpack .../38-make_4.3-4.1_amd64.deb ...
Unpacking make (4.3-4.1) ...
Selecting previously unselected package libdpkg-perl.
Preparing to unpack .../39-libdpkg-perl_1.20.12_all.deb ...
Unpacking libdpkg-perl (1.20.12) ...
Selecting previously unselected package patch.
Preparing to unpack .../40-patch_2.7.6-7_amd64.deb ...
Unpacking patch (2.7.6-7) ...
Selecting previously unselected package dpkg-dev.
Preparing to unpack .../41-dpkg-dev_1.20.12_all.deb ...
Unpacking dpkg-dev (1.20.12) ...
Selecting previously unselected package build-essential.
Preparing to unpack .../42-build-essential_12.9_amd64.deb ...
Unpacking build-essential (12.9) ...
Selecting previously unselected package libfakeroot:amd64.
Preparing to unpack .../43-libfakeroot_1.25.3-1.1_amd64.deb ...
Unpacking libfakeroot:amd64 (1.25.3-1.1) ...
Selecting previously unselected package fakeroot.
Preparing to unpack .../44-fakeroot_1.25.3-1.1_amd64.deb ...
Unpacking fakeroot (1.25.3-1.1) ...
Selecting previously unselected package fonts-dejavu-core.
Preparing to unpack .../45-fonts-dejavu-core_2.37-2_all.deb ...
Unpacking fonts-dejavu-core (2.37-2) ...
Selecting previously unselected package fontconfig-config.
Preparing to unpack .../46-fontconfig-config_2.13.1-4.2_all.deb ...
Unpacking fontconfig-config (2.13.1-4.2) ...
Selecting previously unselected package liberror-perl.
Preparing to unpack .../47-liberror-perl_0.17029-1_all.deb ...
Unpacking liberror-perl (0.17029-1) ...
Selecting previously unselected package git-man.
Preparing to unpack .../48-git-man_1%3a2.30.2-1+deb11u2_all.deb ...
Unpacking git-man (1:2.30.2-1+deb11u2) ...
Selecting previously unselected package git.
Preparing to unpack .../49-git_1%3a2.30.2-1+deb11u2_amd64.deb ...
Unpacking git (1:2.30.2-1+deb11u2) ...
Selecting previously unselected package javascript-common.
Preparing to unpack .../50-javascript-common_11+nmu1_all.deb ...
Unpacking javascript-common (11+nmu1) ...
Selecting previously unselected package libalgorithm-diff-perl.
Preparing to unpack .../51-libalgorithm-diff-perl_1.201-1_all.deb ...
Unpacking libalgorithm-diff-perl (1.201-1) ...
Selecting previously unselected package libalgorithm-diff-xs-perl.
Preparing to unpack .../52-libalgorithm-diff-xs-perl_0.04-6+b1_amd64.deb ...
Unpacking libalgorithm-diff-xs-perl (0.04-6+b1) ...
Selecting previously unselected package libalgorithm-merge-perl.
Preparing to unpack .../53-libalgorithm-merge-perl_0.08-3_all.deb ...
Unpacking libalgorithm-merge-perl (0.08-3) ...
Selecting previously unselected package libfontconfig1:amd64.
Preparing to unpack .../54-libfontconfig1_2.13.1-4.2_amd64.deb ...
Unpacking libfontconfig1:amd64 (2.13.1-4.2) ...
Selecting previously unselected package libjpeg62-turbo:amd64.
Preparing to unpack .../55-libjpeg62-turbo_1%3a2.0.6-4_amd64.deb ...
Unpacking libjpeg62-turbo:amd64 (1:2.0.6-4) ...
Selecting previously unselected package libdeflate0:amd64.
Preparing to unpack .../56-libdeflate0_1.7-1_amd64.deb ...
Unpacking libdeflate0:amd64 (1.7-1) ...
Selecting previously unselected package libjbig0:amd64.
Preparing to unpack .../57-libjbig0_2.1-3.1+b2_amd64.deb ...
Unpacking libjbig0:amd64 (2.1-3.1+b2) ...
Selecting previously unselected package libwebp6:amd64.
Preparing to unpack .../58-libwebp6_0.6.1-2.1+deb11u1_amd64.deb ...
Unpacking libwebp6:amd64 (0.6.1-2.1+deb11u1) ...
Selecting previously unselected package libtiff5:amd64.
Preparing to unpack .../59-libtiff5_4.2.0-1+deb11u4_amd64.deb ...
Unpacking libtiff5:amd64 (4.2.0-1+deb11u4) ...
Selecting previously unselected package libxau6:amd64.
Preparing to unpack .../60-libxau6_1%3a1.0.9-1_amd64.deb ...
Unpacking libxau6:amd64 (1:1.0.9-1) ...
Selecting previously unselected package libxdmcp6:amd64.
Preparing to unpack .../61-libxdmcp6_1%3a1.1.2-3_amd64.deb ...
Unpacking libxdmcp6:amd64 (1:1.1.2-3) ...
Selecting previously unselected package libxcb1:amd64.
Preparing to unpack .../62-libxcb1_1.14-3_amd64.deb ...
Unpacking libxcb1:amd64 (1.14-3) ...
Selecting previously unselected package libx11-data.
Preparing to unpack .../63-libx11-data_2%3a1.7.2-1+deb11u1_all.deb ...
Unpacking libx11-data (2:1.7.2-1+deb11u1) ...
Selecting previously unselected package libx11-6:amd64.
Preparing to unpack .../64-libx11-6_2%3a1.7.2-1+deb11u1_amd64.deb ...
Unpacking libx11-6:amd64 (2:1.7.2-1+deb11u1) ...
Selecting previously unselected package libxpm4:amd64.
Preparing to unpack .../65-libxpm4_1%3a3.5.12-1.1~deb11u1_amd64.deb ...
Unpacking libxpm4:amd64 (1:3.5.12-1.1~deb11u1) ...
Selecting previously unselected package libgd3:amd64.
Preparing to unpack .../66-libgd3_2.3.0-2_amd64.deb ...
Unpacking libgd3:amd64 (2.3.0-2) ...
Selecting previously unselected package libc-devtools.
Preparing to unpack .../67-libc-devtools_2.31-13+deb11u6_amd64.deb ...
Unpacking libc-devtools (2.31-13+deb11u6) ...
Selecting previously unselected package libexpat1-dev:amd64.
Preparing to unpack .../68-libexpat1-dev_2.2.10-2+deb11u5_amd64.deb ...
Unpacking libexpat1-dev:amd64 (2.2.10-2+deb11u5) ...
Selecting previously unselected package libfile-fcntllock-perl.
Preparing to unpack .../69-libfile-fcntllock-perl_0.22-3+b7_amd64.deb ...
Unpacking libfile-fcntllock-perl (0.22-3+b7) ...
Selecting previously unselected package libjs-jquery.
Preparing to unpack .../70-libjs-jquery_3.5.1+dfsg+~3.5.5-7_all.deb ...
Unpacking libjs-jquery (3.5.1+dfsg+~3.5.5-7) ...
Selecting previously unselected package libjs-underscore.
Preparing to unpack .../71-libjs-underscore_1.9.1~dfsg-3_all.deb ...
Unpacking libjs-underscore (1.9.1~dfsg-3) ...
Selecting previously unselected package libjs-sphinxdoc.
Preparing to unpack .../72-libjs-sphinxdoc_3.4.3-2_all.deb ...
Unpacking libjs-sphinxdoc (3.4.3-2) ...
Selecting previously unselected package libpython3.9-dev:amd64.
Preparing to unpack .../73-libpython3.9-dev_3.9.2-1_amd64.deb ...
Unpacking libpython3.9-dev:amd64 (3.9.2-1) ...
Selecting previously unselected package libpython3-dev:amd64.
Preparing to unpack .../74-libpython3-dev_3.9.2-3_amd64.deb ...
Unpacking libpython3-dev:amd64 (3.9.2-3) ...
Selecting previously unselected package manpages-dev.
Preparing to unpack .../75-manpages-dev_5.10-1_all.deb ...
Unpacking manpages-dev (5.10-1) ...
Selecting previously unselected package python-pip-whl.
Preparing to unpack .../76-python-pip-whl_20.3.4-4+deb11u1_all.deb ...
Unpacking python-pip-whl (20.3.4-4+deb11u1) ...
Selecting previously unselected package zlib1g-dev:amd64.
Preparing to unpack .../77-zlib1g-dev_1%3a1.2.11.dfsg-2+deb11u2_amd64.deb ...
Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-2+deb11u2) ...
Selecting previously unselected package python3.9-dev.
Preparing to unpack .../78-python3.9-dev_3.9.2-1_amd64.deb ...
Unpacking python3.9-dev (3.9.2-1) ...
Selecting previously unselected package python3-lib2to3.
Preparing to unpack .../79-python3-lib2to3_3.9.2-1_all.deb ...
Unpacking python3-lib2to3 (3.9.2-1) ...
Selecting previously unselected package python3-distutils.
Preparing to unpack .../80-python3-distutils_3.9.2-1_all.deb ...
Unpacking python3-distutils (3.9.2-1) ...
Selecting previously unselected package python3-dev.
Preparing to unpack .../81-python3-dev_3.9.2-3_amd64.deb ...
Unpacking python3-dev (3.9.2-3) ...
Selecting previously unselected package python3-setuptools.
Preparing to unpack .../82-python3-setuptools_52.0.0-4_all.deb ...
Unpacking python3-setuptools (52.0.0-4) ...
Selecting previously unselected package python3-wheel.
Preparing to unpack .../83-python3-wheel_0.34.2-1_all.deb ...
Unpacking python3-wheel (0.34.2-1) ...
Selecting previously unselected package python3-pip.
Preparing to unpack .../84-python3-pip_20.3.4-4+deb11u1_all.deb ...
Unpacking python3-pip (20.3.4-4+deb11u1) ...
Setting up javascript-common (11+nmu1) ...
Setting up manpages-dev (5.10-1) ...
Setting up libxau6:amd64 (1:1.0.9-1) ...
Setting up libxdmcp6:amd64 (1:1.1.2-3) ...
Setting up libxcb1:amd64 (1.14-3) ...
Setting up perl-modules-5.32 (5.32.1-4+deb11u2) ...
Setting up binutils-common:amd64 (2.35.2-2) ...
Setting up libdeflate0:amd64 (1.7-1) ...
Setting up linux-libc-dev:amd64 (5.10.179-3) ...
Setting up libctf-nobfd0:amd64 (2.35.2-2) ...
Setting up libgomp1:amd64 (10.2.1-6) ...
Setting up bzip2 (1.0.8-4) ...
Setting up python3-wheel (0.34.2-1) ...
Setting up libjbig0:amd64 (2.1-3.1+b2) ...
Setting up libfakeroot:amd64 (1.25.3-1.1) ...
Setting up libasan6:amd64 (10.2.1-6) ...
Setting up fakeroot (1.25.3-1.1) ...
update-alternatives: using /usr/bin/fakeroot-sysv to provide /usr/bin/fakeroot (fakeroot) in auto mode
Setting up libtirpc-dev:amd64 (1.3.1-1+deb11u1) ...
Setting up libjpeg62-turbo:amd64 (1:2.0.6-4) ...
Setting up libx11-data (2:1.7.2-1+deb11u1) ...
Setting up make (4.3-4.1) ...
Setting up libmpfr6:amd64 (4.1.0-3) ...
Setting up libquadmath0:amd64 (10.2.1-6) ...
Setting up libmpc3:amd64 (1.2.0-1) ...
Setting up libatomic1:amd64 (10.2.1-6) ...
Setting up patch (2.7.6-7) ...
Setting up libwebp6:amd64 (0.6.1-2.1+deb11u1) ...
Setting up fonts-dejavu-core (2.37-2) ...
Setting up libgdbm-compat4:amd64 (1.19-2) ...
Setting up libperl5.32:amd64 (5.32.1-4+deb11u2) ...
Setting up libubsan1:amd64 (10.2.1-6) ...
Setting up libnsl-dev:amd64 (1.3.0-2) ...
Setting up libcrypt-dev:amd64 (1:4.4.18-4) ...
Setting up git-man (1:2.30.2-1+deb11u2) ...
Setting up libx11-6:amd64 (2:1.7.2-1+deb11u1) ...
Setting up python-pip-whl (20.3.4-4+deb11u1) ...
Setting up libtiff5:amd64 (4.2.0-1+deb11u4) ...
Setting up libjs-jquery (3.5.1+dfsg+~3.5.5-7) ...
Setting up libbinutils:amd64 (2.35.2-2) ...
Setting up libisl23:amd64 (0.23-1) ...
Setting up libc-dev-bin (2.31-13+deb11u6) ...
Setting up python3-lib2to3 (3.9.2-1) ...
Setting up libcc1-0:amd64 (10.2.1-6) ...
Setting up liblocale-gettext-perl (1.07-4+b1) ...
Setting up liblsan0:amd64 (10.2.1-6) ...
Setting up cpp-10 (10.2.1-6) ...
Setting up libitm1:amd64 (10.2.1-6) ...
Setting up libjs-underscore (1.9.1~dfsg-3) ...
Setting up libtsan0:amd64 (10.2.1-6) ...
Setting up libctf0:amd64 (2.35.2-2) ...
Setting up python3-distutils (3.9.2-1) ...
Setting up python3-setuptools (52.0.0-4) ...
Setting up libxpm4:amd64 (1:3.5.12-1.1~deb11u1) ...
Setting up fontconfig-config (2.13.1-4.2) ...
Setting up libgcc-10-dev:amd64 (10.2.1-6) ...
Setting up perl (5.32.1-4+deb11u2) ...
Setting up python3-pip (20.3.4-4+deb11u1) ...
Setting up libjs-sphinxdoc (3.4.3-2) ...
Setting up libdpkg-perl (1.20.12) ...
Setting up cpp (4:10.2.1-1) ...
Setting up libc6-dev:amd64 (2.31-13+deb11u6) ...
Setting up libfontconfig1:amd64 (2.13.1-4.2) ...
Setting up binutils-x86-64-linux-gnu (2.35.2-2) ...
Setting up libstdc++-10-dev:amd64 (10.2.1-6) ...
Setting up libfile-fcntllock-perl (0.22-3+b7) ...
Setting up libalgorithm-diff-perl (1.201-1) ...
Setting up binutils (2.35.2-2) ...
Setting up dpkg-dev (1.20.12) ...
Setting up liberror-perl (0.17029-1) ...
Setting up libexpat1-dev:amd64 (2.2.10-2+deb11u5) ...
Setting up libgd3:amd64 (2.3.0-2) ...
Setting up git (1:2.30.2-1+deb11u2) ...
Setting up gcc-10 (10.2.1-6) ...
Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-2+deb11u2) ...
Setting up libalgorithm-diff-xs-perl (0.04-6+b1) ...
Setting up libc-devtools (2.31-13+deb11u6) ...
Setting up libalgorithm-merge-perl (0.08-3) ...
Setting up g++-10 (10.2.1-6) ...
Setting up libpython3.9-dev:amd64 (3.9.2-1) ...
Setting up gcc (4:10.2.1-1) ...
Setting up g++ (4:10.2.1-1) ...
update-alternatives: using /usr/bin/g++ to provide /usr/bin/c++ (c++) in auto mode
Setting up python3.9-dev (3.9.2-1) ...
Setting up build-essential (12.9) ...
Setting up libpython3-dev:amd64 (3.9.2-3) ...
Setting up python3-dev (3.9.2-3) ...
Processing triggers for man-db (2.9.4-2) ...
Processing triggers for libc-bin (2.31-13+deb11u6) ...
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$ pip3 install --upgrade pip
Requirement already satisfied: pip in /usr/lib/python3/dist-packages (20.3.4)
Collecting pip
Downloading pip-23.2.1-py3-none-any.whl (2.1 MB)
|████████████████████████████████| 2.1 MB 7.3 MB/s
Installing collected packages: pip
WARNING: The scripts pip, pip3, pip3.11 and pip3.9 are installed in '/home/student-04-717d1aa28d01/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-23.2.1
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$ pip3 install google-cloud-bigquery
Defaulting to user installation because normal site-packages is not writeable
Collecting google-cloud-bigquery
Obtaining dependency information for google-cloud-bigquery from https://files.pythonhosted.org/packages/cc/6a/d0ef792288f2fa2cfea80899a82de302b3332dfda41984fe114e2cfbf700/google_cloud_bigquery-3.11.4-py2.py3-none-any.whl.metadata
Downloading google_cloud_bigquery-3.11.4-py2.py3-none-any.whl.metadata (8.5 kB)
Collecting grpcio<2.0dev,>=1.47.0 (from google-cloud-bigquery)
Obtaining dependency information for grpcio<2.0dev,>=1.47.0 from https://files.pythonhosted.org/packages/f5/f6/57fbd39af17aaae321109411ef2faf121768473ebc1bbf3694b06d3282c8/grpcio-1.56.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Downloading grpcio-1.56.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.0 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5 (from google-cloud-bigquery)
Obtaining dependency information for google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5 from https://files.pythonhosted.org/packages/6e/c4/c3cd048b6cbeba8d9ae50dd7643ac065b85237338aa7501b0efae91eb4d9/google_api_core-2.11.1-py3-none-any.whl.metadata
Downloading google_api_core-2.11.1-py3-none-any.whl.metadata (2.7 kB)
Collecting proto-plus<2.0.0dev,>=1.15.0 (from google-cloud-bigquery)
Obtaining dependency information for proto-plus<2.0.0dev,>=1.15.0 from https://files.pythonhosted.org/packages/36/5b/e02636d221917d6fa2a61289b3f16002eb4c93d51c0191ac8e896d527182/proto_plus-1.22.3-py3-none-any.whl.metadata
Downloading proto_plus-1.22.3-py3-none-any.whl.metadata (2.2 kB)
Collecting google-cloud-core<3.0.0dev,>=1.6.0 (from google-cloud-bigquery)
Obtaining dependency information for google-cloud-core<3.0.0dev,>=1.6.0 from https://files.pythonhosted.org/packages/a2/40/02045f776fdb6e44194f34b6375a26ce8a61bd9bd03cd8930ed91cf51a62/google_cloud_core-2.3.3-py2.py3-none-any.whl.metadata
Downloading google_cloud_core-2.3.3-py2.py3-none-any.whl.metadata (2.5 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery)
Downloading google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.7/77.7 kB 5.4 MB/s eta 0:00:00
Collecting packaging>=20.0.0 (from google-cloud-bigquery)
Downloading packaging-23.1-py3-none-any.whl (48 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.9/48.9 kB 9.9 MB/s eta 0:00:00
Collecting protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5 (from google-cloud-bigquery)
Obtaining dependency information for protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5 from https://files.pythonhosted.org/packages/01/cb/445b3e465abdb8042a41957dc8f60c54620dc7540dbcf9b458a921531ca2/protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl.metadata
Downloading protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl.metadata (540 bytes)
Collecting python-dateutil<3.0dev,>=2.7.2 (from google-cloud-bigquery)
Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 kB 19.0 MB/s eta 0:00:00
Requirement already satisfied: requests<3.0.0dev,>=2.21.0 in /usr/lib/python3/dist-packages (from google-cloud-bigquery) (2.25.1)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-bigquery)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Downloading googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting google-auth<3.0.dev0,>=2.14.1 (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-bigquery)
Obtaining dependency information for google-auth<3.0.dev0,>=2.14.1 from https://files.pythonhosted.org/packages/9c/8d/bff87fc722553a5691d8514da5523c23547f3894189ba03b57592e37bdc2/google_auth-2.22.0-py2.py3-none-any.whl.metadata
Downloading google_auth-2.22.0-py2.py3-none-any.whl.metadata (4.2 kB)
Collecting grpcio-status<2.0.dev0,>=1.33.2 (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-bigquery)
Obtaining dependency information for grpcio-status<2.0.dev0,>=1.33.2 from https://files.pythonhosted.org/packages/ef/16/3018689d96918e9c4c7407adf96b721df4d6748ba65db82c5eaa63564335/grpcio_status-1.56.2-py3-none-any.whl.metadata
Downloading grpcio_status-1.56.2-py3-none-any.whl.metadata (1.3 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery)
Downloading google_crc32c-1.5.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil<3.0dev,>=2.7.2->google-cloud-bigquery) (1.16.0)
Collecting cachetools<6.0,>=2.0.0 (from google-auth<3.0.dev0,>=2.14.1->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-bigquery)
Obtaining dependency information for cachetools<6.0,>=2.0.0 from https://files.pythonhosted.org/packages/a9/c9/c8a7710f2cedcb1db9224fdd4d8307c9e48cbddc46c18b515fefc0f1abbe/cachetools-5.3.1-py3-none-any.whl.metadata
Downloading cachetools-5.3.1-py3-none-any.whl.metadata (5.2 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3.0.dev0,>=2.14.1->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-bigquery)
Downloading pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 181.3/181.3 kB 35.5 MB/s eta 0:00:00
Collecting rsa<5,>=3.1.4 (from google-auth<3.0.dev0,>=2.14.1->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-bigquery)
Downloading rsa-4.9-py3-none-any.whl (34 kB)
Requirement already satisfied: urllib3<2.0 in /usr/lib/python3/dist-packages (from google-auth<3.0.dev0,>=2.14.1->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-bigquery) (1.26.5)
Collecting pyasn1<0.6.0,>=0.4.6 (from pyasn1-modules>=0.2.1->google-auth<3.0.dev0,>=2.14.1->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-bigquery)
Downloading pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 83.9/83.9 kB 18.3 MB/s eta 0:00:00
Downloading google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 219.6/219.6 kB 38.2 MB/s eta 0:00:00
Downloading google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Downloading grpcio-1.56.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.2/5.2 MB 51.1 MB/s eta 0:00:00
Downloading proto_plus-1.22.3-py3-none-any.whl (48 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.1/48.1 kB 10.2 MB/s eta 0:00:00
Downloading protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 304.5/304.5 kB 43.7 MB/s eta 0:00:00
Downloading google_api_core-2.11.1-py3-none-any.whl (120 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 120.5/120.5 kB 23.7 MB/s eta 0:00:00
Downloading google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 181.8/181.8 kB 34.6 MB/s eta 0:00:00
Downloading googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 227.6/227.6 kB 38.1 MB/s eta 0:00:00
Downloading grpcio_status-1.56.2-py3-none-any.whl (5.1 kB)
Downloading cachetools-5.3.1-py3-none-any.whl (9.3 kB)
Installing collected packages: python-dateutil, pyasn1, protobuf, packaging, grpcio, google-crc32c, cachetools, rsa, pyasn1-modules, proto-plus, googleapis-common-protos, google-resumable-media, grpcio-status, google-auth, google-api-core, google-cloud-core, google-cloud-bigquery
WARNING: The scripts pyrsa-decrypt, pyrsa-encrypt, pyrsa-keygen, pyrsa-priv2pub, pyrsa-sign and pyrsa-verify are installed in '/home/student-04-717d1aa28d01/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed cachetools-5.3.1 google-api-core-2.11.1 google-auth-2.22.0 google-cloud-bigquery-3.11.4 google-cloud-core-2.3.3 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 grpcio-1.56.2 grpcio-status-1.56.2 packaging-23.1 proto-plus-1.22.3 protobuf-4.23.4 pyasn1-0.5.0 pyasn1-modules-0.3.0 python-dateutil-2.8.2 rsa-4.9
student-04-717d1aa28d01@bigquery-instance:~$ pip3 install pyarrow
Defaulting to user installation because normal site-packages is not writeable
Collecting pyarrow
Obtaining dependency information for pyarrow from https://files.pythonhosted.org/packages/54/a2/5976df95323c4ca2b7baba31cb7a2a61a17461706043239d38a8e9dc281e/pyarrow-12.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Downloading pyarrow-12.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.0 kB)
Collecting numpy>=1.16.6 (from pyarrow)
Obtaining dependency information for numpy>=1.16.6 from https://files.pythonhosted.org/packages/69/1f/c95b1108a9972a52d7b1b63ed8ca70466b59b8c1811bd121f1e667cc45d8/numpy-1.25.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Downloading numpy-1.25.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.6 kB)
Downloading pyarrow-12.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (39.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 39.0/39.0 MB 24.2 MB/s eta 0:00:00
Downloading numpy-1.25.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.3/18.3 MB 49.3 MB/s eta 0:00:00
Installing collected packages: numpy, pyarrow
WARNING: The scripts f2py, f2py3 and f2py3.9 are installed in '/home/student-04-717d1aa28d01/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed numpy-1.25.2 pyarrow-12.0.1
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$ pip3 install pandas
Defaulting to user installation because normal site-packages is not writeable
Collecting pandas
Obtaining dependency information for pandas from https://files.pythonhosted.org/packages/9e/0d/91a9fd2c202f2b1d97a38ab591890f86480ecbb596cbc56d035f6f23fdcc/pandas-2.0.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Downloading pandas-2.0.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (18 kB)
Requirement already satisfied: python-dateutil>=2.8.2 in ./.local/lib/python3.9/site-packages (from pandas) (2.8.2)
Collecting pytz>=2020.1 (from pandas)
Downloading pytz-2023.3-py2.py3-none-any.whl (502 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 502.3/502.3 kB 13.8 MB/s eta 0:00:00
Collecting tzdata>=2022.1 (from pandas)
Downloading tzdata-2023.3-py2.py3-none-any.whl (341 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 341.8/341.8 kB 48.7 MB/s eta 0:00:00
Requirement already satisfied: numpy>=1.20.3 in ./.local/lib/python3.9/site-packages (from pandas) (1.25.2)
Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil>=2.8.2->pandas) (1.16.0)
Downloading pandas-2.0.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.4 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.4/12.4 MB 51.1 MB/s eta 0:00:00
Installing collected packages: pytz, tzdata, pandas
Successfully installed pandas-2.0.3 pytz-2023.3 tzdata-2023.3
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$ pip3 install db-dtypes
Defaulting to user installation because normal site-packages is not writeable
Collecting db-dtypes
Downloading db_dtypes-1.1.1-py2.py3-none-any.whl (14 kB)
Requirement already satisfied: packaging>=17.0 in ./.local/lib/python3.9/site-packages (from db-dtypes) (23.1)
Requirement already satisfied: pandas>=0.24.2 in ./.local/lib/python3.9/site-packages (from db-dtypes) (2.0.3)
Requirement already satisfied: pyarrow>=3.0.0 in ./.local/lib/python3.9/site-packages (from db-dtypes) (12.0.1)
Requirement already satisfied: numpy>=1.16.6 in ./.local/lib/python3.9/site-packages (from db-dtypes) (1.25.2)
Requirement already satisfied: python-dateutil>=2.8.2 in ./.local/lib/python3.9/site-packages (from pandas>=0.24.2->db-dtypes) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in ./.local/lib/python3.9/site-packages (from pandas>=0.24.2->db-dtypes) (2023.3)
Requirement already satisfied: tzdata>=2022.1 in ./.local/lib/python3.9/site-packages (from pandas>=0.24.2->db-dtypes) (2023.3)
Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil>=2.8.2->pandas>=0.24.2->db-dtypes) (1.16.0)
Installing collected packages: db-dtypes
Successfully installed db-dtypes-1.1.1
student-04-717d1aa28d01@bigquery-instance:~$ vi query.py
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$ sed -i -e "s/qwiklabs-gcp-03-f3d72bb5968f/$(gcloud config get-value project)/g" query.py
student-04-717d1aa28d01@bigquery-instance:~$ vi query.py
student-04-717d1aa28d01@bigquery-instance:~$ rm query.py
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$ echo "
from google.auth import compute_engine
from google.cloud import bigquery
credentials = compute_engine.Credentials(
service_account_email='YOUR_SERVICE_ACCOUNT')
query = '''
SELECT
year,
COUNT(1) as num_babies
FROM
publicdata.samples.natality
WHERE
year > 2000
GROUP BY
year
'''
client = bigquery.Client(
project='qwiklabs-gcp-03-f3d72bb5968f',
credentials=credentials)
print(client.query(query).to_dataframe())
" > query.py
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$ cat query.py
from google.auth import compute_engine
from google.cloud import bigquery
credentials = compute_engine.Credentials(
service_account_email='YOUR_SERVICE_ACCOUNT')
query = '''
SELECT
year,
COUNT(1) as num_babies
FROM
publicdata.samples.natality
WHERE
year > 2000
GROUP BY
year
'''
client = bigquery.Client(
project='qwiklabs-gcp-03-f3d72bb5968f',
credentials=credentials)
print(client.query(query).to_dataframe())
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$ sed -i -e "s/qwiklabs-gcp-03-f3d72bb5968f/$(gcloud config get-value project)/g" query.py
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$ cat query.py
from google.auth import compute_engine
from google.cloud import bigquery
credentials = compute_engine.Credentials(
service_account_email='YOUR_SERVICE_ACCOUNT')
query = '''
SELECT
year,
COUNT(1) as num_babies
FROM
publicdata.samples.natality
WHERE
year > 2000
GROUP BY
year
'''
client = bigquery.Client(
project='qwiklabs-gcp-03-f3d72bb5968f',
credentials=credentials)
print(client.query(query).to_dataframe())
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$ sed -i -e "s/YOUR_SERVICE_ACCOUNT/bigquery-qwiklab@$(gcloud config get-value project).iam.gserviceaccount.com/g" query.py
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$ cat query.py
from google.auth import compute_engine
from google.cloud import bigquery
credentials = compute_engine.Credentials(
service_account_email='bigquery-qwiklab@qwiklabs-gcp-03-f3d72bb5968f.iam.gserviceaccount.com')
query = '''
SELECT
year,
COUNT(1) as num_babies
FROM
publicdata.samples.natality
WHERE
year > 2000
GROUP BY
year
'''
client = bigquery.Client(
project='qwiklabs-gcp-03-f3d72bb5968f',
credentials=credentials)
print(client.query(query).to_dataframe())
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$
student-04-717d1aa28d01@bigquery-instance:~$ python3 query.py
year num_babies
0 2006 4273225
1 2004 4118907
2 2005 4145619
3 2002 4027376
4 2007 4324008
5 2008 4255156
6 2001 4031531
7 2003 4096092
student-04-717d1aa28d01@bigquery-instance:~$ history
1 sudo apt-get update
2 sudo apt-get install -y git python3-pip
3 pip3 install --upgrade pip
4 pip3 install google-cloud-bigquery
5 pip3 install pyarrow
6 pip3 install pandas
7 pip3 install db-dtypes
8 vi query.py
9 sed -i -e "s/qwiklabs-gcp-03-f3d72bb5968f/$(gcloud config get-value project)/g" query.py
10 vi query.py
11 rm query.py
12 echo "
from google.auth import compute_engine
from google.cloud import bigquery
credentials = compute_engine.Credentials(
service_account_email='YOUR_SERVICE_ACCOUNT')
query = '''
SELECT
year,
COUNT(1) as num_babies
FROM
publicdata.samples.natality
WHERE
year > 2000
GROUP BY
year
'''
client = bigquery.Client(
project='qwiklabs-gcp-03-f3d72bb5968f',
credentials=credentials)
print(client.query(query).to_dataframe())
" > query.py
13 cat query.py
14 sed -i -e "s/qwiklabs-gcp-03-f3d72bb5968f/$(gcloud config get-value project)/g" query.py
15 cat query.py
16 sed -i -e "s/YOUR_SERVICE_ACCOUNT/bigquery-qwiklab@$(gcloud config get-value project).iam.gserviceaccount.com/g" query.py
17 cat query.py
18 python3 query.py
19 history
student-04-717d1aa28d01@bigquery-instance:~$
No comments:
Post a Comment