If you want to utilize BigQuery’s INFORMATION_SCHEMA tables as a source, it’s relatively straightforward. You just set it as any other table way with a region prefix, ensuring you include the region prefix.
INFORMATION_SCHEMA in source.yml
sources:
- name: gcp
schema: region-us.INFORMATION_SCHEMA
tables:
- JOBS_BY_FOLDER
- JOBS_BY_PROJECT
- ...
- JOBS
Issues, errors?
In generated documentation with dbt docs
everything seems to be in place.
Although, each execution produces DatabaseError about incorrect project_id
format. I didn’t find any other alternative way how to write it down.
Solution
The workaround, I came up with, is executing compilation separately before as:
dbt compile
dbt docs generate --no-compile
Don’t know whether this is graceful enough, but at least don’t produce hundreds of errors in BigQuery’s job history. Especially helpful, if you generate documentation very often.