Ir para o conteúdo

CLI API

Esta API é composta de comandos para gerenciamento e inserção de dados no Google Cloud.

Os comandos disponíveis dentro de cli atualmente são:

  • config, que é utilizado para atualizar seus dados de configuração e gerenciar templates
  • dataset, que permite gerenciar datasets no BigQuery (criar, modificar, publicar , atualizar e deletar)
  • download, que permite baixar/salvar queries de tabelas do BigQuery na sua máquina local
  • storage, que permite gerenciar seu Storage no BigQuery (criar e subir arquivos)
  • table, que permite gerenciar tabelas no BigQuery (criar, modificar, publicar , atualizar e deletar)

Toda documentação do código abaixo está em inglês

cli

Usage:

cli [OPTIONS] COMMAND [ARGS]...

Options:

  --templates TEXT      Templates path
  --bucket_name TEXT    Project bucket name
  --metadata_path TEXT  Folder to store metadata

config

Usage:

cli config [OPTIONS] COMMAND [ARGS]...

overwrite_cli_config

Overwrite current configuration

Usage:

cli config overwrite_cli_config [OPTIONS]

refresh_template

Overwrite current templates

Usage:

cli config refresh_template [OPTIONS]

dataset

Usage:

cli dataset [OPTIONS] COMMAND [ARGS]...

create

Create dataset on BigQuery

Usage:

cli dataset create [OPTIONS] DATASET_ID

Options:

  -m, --mode TEXT   What datasets to create [all|staging|prod]
  --if_exists TEXT  [raise|update|replace|pass] if dataset alread exists

delete

Delete dataset

Usage:

cli dataset delete [OPTIONS] DATASET_ID

Options:

  -m, --mode TEXT  What datasets to create [all|staging|prod]

init

Initialize metadata files of dataset

Usage:

cli dataset init [OPTIONS] DATASET_ID

Options:

  --replace  Whether to replace current metadata files

publicize

Make a dataset public

Usage:

cli dataset publicize [OPTIONS] DATASET_ID

update

Update dataset on BigQuery

Usage:

cli dataset update [OPTIONS] DATASET_ID

Options:

  -m, --mode TEXT  What datasets to create [all|staging|prod]

download

Download data. You can add extra arguments accepted by pandas.to_csv.

Examples: --delimiter='|', --index=False

Usage:

cli download [OPTIONS] SAVEPATH

Options:

  --dataset_id TEXT          Dataset_id, enter with table_id to download table
  --table_id TEXT            Table_id, enter with dataset_id to download table
  --query TEXT               A SQL Standard query to download data from
                             BigQuery

  --query_project_id TEXT    Which project the table lives. You can change
                             this you want to query different projects.

  --billing_project_id TEXT  Project that will be billed. Find your Project ID
                             here https://console.cloud.google.com/projectsele
                             ctor2/home/dashboard

  --limit TEXT               Number of rows returned

storage

Usage:

cli storage [OPTIONS] COMMAND [ARGS]...

init

Create bucket and initial folders

Usage:

cli storage init [OPTIONS]

Options:

  --bucket_name TEXT        Bucket name
  --replace                 Whether to replace current bucket files
  --very-sure / --not-sure  Are you sure that you want to replace current
                            bucket files?

upload

Upload file to bucket

Usage:

cli storage upload [OPTIONS] DATASET_ID TABLE_ID FILEPATH

Options:

  -m, --mode TEXT    [raw|staging] where to save the file  [required]
  --partitions TEXT  Data partition as `value=key/value2=key2`
  --if_exists TEXT   [raise|replace|pass] if file alread exists

table

Usage:

cli table [OPTIONS] COMMAND [ARGS]...

append

Append new data to existing table

Usage:

cli table append [OPTIONS] DATASET_ID TABLE_ID FILEPATH

Options:

  --partitions TEXT  Data partition as `value=key/value2=key2`
  --if_exists TEXT   [raise|replace|pass] if file alread exists

create

Create stagging table in BigQuery

Usage:

cli table create [OPTIONS] DATASET_ID TABLE_ID

Options:

  -p, --path PATH           Path of data folder or file.
  --job_config_params TEXT  File to advanced load config params
  --partitioned             [True|False] whether folder has partitions
  --if_exists TEXT          [raise|replace|pass] actions if table exists
  --force_dataset TEXT      Whether to automatically create the dataset
                            folders and in BigQuery

delete

Delete BigQuery table

Usage:

cli table delete [OPTIONS] DATASET_ID TABLE_ID

Options:

  --mode TEXT  Which table to delete [all|prod|staging]  [required]

init

Create metadata files

Usage:

cli table init [OPTIONS] DATASET_ID TABLE_ID

Options:

  --data_sample_path PATH  Sample data used to pre-fill metadata
  --if_exists TEXT         [raise|replace|pass] actions if table folder exists

publish

Publish staging table to prod

Usage:

cli table publish [OPTIONS] DATASET_ID TABLE_ID

Options:

  --if_exists TEXT  [raise|replace] actions if table exists

update

Update tables in BigQuery

Usage:

cli table update [OPTIONS] DATASET_ID TABLE_ID

Options:

  --mode TEXT  Choose a table from a dataset to update [all|staging|prod]