feat(DENG-9188) Add BigConfig checks to targeted stable tables

[phil-lee70]

Description

This PR adds the feature to add BigConfig monitoring checks (freshness and volume) to a targeted set of stable tables.

Approach - apply this change to an asset that is currently not in use
This PR will create the metadata and bigconfig yaml files for moz-fx-data-shared-prod.mozilla_vpn_external.waitlist_v1. This table has not been queried in the last 6 months and does not have any downstream tables or Looker Explores.

Output:

TODO:
Confirm if this may impact other configurations in BigQuery unexpectedly.

Related Tickets & Documents

• DENG-9188

Reviewer, please follow this checklist

[dataops-ci-bot]

Integration report for "Update README.md"

sql.diff

Click to expand!

diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_glean_usage.py /tmp/workspace/generated-sql/dags/bqetl_glean_usage.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_glean_usage.py	2025-11-17 16:10:54.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_glean_usage.py	2025-11-17 16:19:43.000000000 +0000
@@ -63,8 +63,6 @@
 
     task_group_debug_ping_view = TaskGroup("debug_ping_view")
 
-    task_group_experimenter_backend = TaskGroup("experimenter_backend")
-
     task_group_experimenter_cirrus = TaskGroup("experimenter_cirrus")
 
     task_group_fenix = TaskGroup("fenix")
@@ -145,6 +143,10 @@
         "subscription_platform_backend"
     )
 
+    task_group_subscription_platform_backend_cirrus = TaskGroup(
+        "subscription_platform_backend_cirrus"
+    )
+
     task_group_syncstorage = TaskGroup("syncstorage")
 
     task_group_thunderbird_android = TaskGroup("thunderbird_android")
@@ -366,54 +368,6 @@
         task_group=task_group_bedrock,
     )
 
-    bigeye__experimenter_backend_derived__baseline_clients_daily__v1 = bigquery_bigeye_check(
-        task_id="bigeye__experimenter_backend_derived__baseline_clients_daily__v1",
-        table_id="moz-fx-data-shared-prod.experimenter_backend_derived.baseline_clients_daily_v1",
-        warehouse_id="1939",
-        owner="[email protected]",
-        email=["[email protected]", "[email protected]"],
-        depends_on_past=False,
-        execution_timeout=datetime.timedelta(hours=1),
-        retries=1,
-        task_group=task_group_experimenter_backend,
-    )
-
-    bigeye__experimenter_backend_derived__baseline_clients_first_seen__v1 = bigquery_bigeye_check(
-        task_id="bigeye__experimenter_backend_derived__baseline_clients_first_seen__v1",
-        table_id="moz-fx-data-shared-prod.experimenter_backend_derived.baseline_clients_first_seen_v1",
-        warehouse_id="1939",
-        owner="[email protected]",
-        email=["[email protected]", "[email protected]"],
-        depends_on_past=False,
-        execution_timeout=datetime.timedelta(hours=1),
-        retries=1,
-        task_group=task_group_experimenter_backend,
-    )
-
-    bigeye__experimenter_backend_derived__baseline_clients_last_seen__v1 = bigquery_bigeye_check(
-        task_id="bigeye__experimenter_backend_derived__baseline_clients_last_seen__v1",
-        table_id="moz-fx-data-shared-prod.experimenter_backend_derived.baseline_clients_last_seen_v1",
-        warehouse_id="1939",
-        owner="[email protected]",
-        email=["[email protected]", "[email protected]"],
-        depends_on_past=False,
-        execution_timeout=datetime.timedelta(hours=1),
-        retries=1,
-        task_group=task_group_experimenter_backend,
-    )
-
-    bigeye__experimenter_backend_derived__metrics_clients_daily__v1 = bigquery_bigeye_check(
-        task_id="bigeye__experimenter_backend_derived__metrics_clients_daily__v1",
-        table_id="moz-fx-data-shared-prod.experimenter_backend_derived.metrics_clients_daily_v1",
-        warehouse_id="1939",
-        owner="[email protected]",
-        email=["[email protected]", "[email protected]"],
-        depends_on_past=False,
-        execution_timeout=datetime.timedelta(hours=1),
-        retries=1,
-        task_group=task_group_experimenter_backend,
-    )
-
     bigeye__experimenter_cirrus_derived__baseline_clients_daily__v1 = bigquery_bigeye_check(
         task_id="bigeye__experimenter_cirrus_derived__baseline_clients_daily__v1",
         table_id="moz-fx-data-shared-prod.experimenter_cirrus_derived.baseline_clients_daily_v1",
@@ -2617,20 +2571,6 @@
         task_group=task_group_burnham,
     )
 
-    checks__fail_experimenter_backend_derived__baseline_clients_last_seen__v1 = bigquery_dq_check(
-        task_id="checks__fail_experimenter_backend_derived__baseline_clients_last_seen__v1",
-        source_table="baseline_clients_last_seen_v1",
-        dataset_id="experimenter_backend_derived",
-        project_id="moz-fx-data-shared-prod",
-        is_dq_check_fail=True,
-        owner="[email protected]",
-        email=["[email protected]", "[email protected]"],
-        depends_on_past=False,
-        parameters=["submission_date:DATE:{{ds}}"],
-        retries=0,
-        task_group=task_group_experimenter_backend,
-    )
-
     checks__fail_experimenter_cirrus_derived__baseline_clients_last_seen__v1 = bigquery_dq_check(
         task_id="checks__fail_experimenter_cirrus_derived__baseline_clients_last_seen__v1",
         source_table="baseline_clients_last_seen_v1",
@@ -3415,73 +3355,6 @@
         task_group=task_group_debug_ping_view,
     )
 
-    experimenter_backend_derived__baseline_clients_daily__v1 = bigquery_etl_query(
-        task_id="experimenter_backend_derived__baseline_clients_daily__v1",
-        destination_table="baseline_clients_daily_v1",
-        dataset_id="experimenter_backend_derived",
-        project_id="moz-fx-data-shared-prod",
-        owner="[email protected]",
-        email=["[email protected]", "[email protected]"],
-        date_partition_parameter="submission_date",
-        depends_on_past=False,
-        task_group=task_group_experimenter_backend,
-    )
-
-    experimenter_backend_derived__baseline_clients_first_seen__v1 = bigquery_etl_query(
-        task_id="experimenter_backend_derived__baseline_clients_first_seen__v1",
-        destination_table="baseline_clients_first_seen_v1",
-        dataset_id="experimenter_backend_derived",
-        project_id="moz-fx-data-shared-prod",
-        owner="[email protected]",
-        email=["[email protected]", "[email protected]"],
-        date_partition_parameter=None,
-        depends_on_past=True,
-        parameters=["submission_date:DATE:{{ds}}"],
-        task_group=task_group_experimenter_backend,
-    )
-
-    experimenter_backend_derived__baseline_clients_last_seen__v1 = bigquery_etl_query(
-        task_id="experimenter_backend_derived__baseline_clients_last_seen__v1",
-        destination_table="baseline_clients_last_seen_v1",
-        dataset_id="experimenter_backend_derived",
-        project_id="moz-fx-data-shared-prod",
-        owner="[email protected]",
-        email=["[email protected]", "[email protected]"],
-        date_partition_parameter="submission_date",
-        depends_on_past=True,
-        task_group=task_group_experimenter_backend,
-    )
-
-    experimenter_backend_derived__events_stream__v1 = bigquery_etl_query(
-        task_id="experimenter_backend_derived__events_stream__v1",
-        destination_table="events_stream_v1",
-        dataset_id="experimenter_backend_derived",
-        project_id="moz-fx-data-shared-prod",
-        owner="[email protected]",
-        email=[
-            "[email protected]",
-            "[email protected]",
-            "[email protected]",
-            "[email protected]",
-        ],
-        date_partition_parameter="submission_date",
-        depends_on_past=False,
-        arguments=["--billing-project", "moz-fx-data-backfill-2"],
-        task_group=task_group_experimenter_backend,
-    )
-
-    experimenter_backend_derived__metrics_clients_daily__v1 = bigquery_etl_query(
-        task_id="experimenter_backend_derived__metrics_clients_daily__v1",
-        destination_table="metrics_clients_daily_v1",
-        dataset_id="experimenter_backend_derived",
-        project_id="moz-fx-data-shared-prod",
-        owner="[email protected]",
-        email=["[email protected]", "[email protected]"],
-        date_partition_parameter="submission_date",
-        depends_on_past=False,
-        task_group=task_group_experimenter_backend,
-    )
-
     experimenter_cirrus_derived__baseline_clients_daily__v1 = bigquery_etl_query(
         task_id="experimenter_cirrus_derived__baseline_clients_daily__v1",
         destination_table="baseline_clients_daily_v1",
@@ -6952,6 +6825,26 @@
         task_group=task_group_relay_backend,
     )
 
+    subscription_platform_backend_cirrus_derived__events_stream__v1 = (
+        bigquery_etl_query(
+            task_id="subscription_platform_backend_cirrus_derived__events_stream__v1",
+            destination_table="events_stream_v1",
+            dataset_id="subscription_platform_backend_cirrus_derived",
+            project_id="moz-fx-data-shared-prod",
+            owner="[email protected]",
+            email=[
+                "[email protected]",
+                "[email protected]",
+                "[email protected]",
+                "[email protected]",
+            ],
+            date_partition_parameter="submission_date",
+            depends_on_past=False,
+            arguments=["--billing-project", "moz-fx-data-backfill-2"],
+            task_group=task_group_subscription_platform_backend_cirrus,
+        )
+    )
+
     subscription_platform_backend_derived__events_stream__v1 = bigquery_etl_query(
         task_id="subscription_platform_backend_derived__events_stream__v1",
         destination_table="events_stream_v1",
@@ -7148,22 +7041,6 @@
 
     bedrock_derived__events_stream__v1.set_upstream(wait_for_copy_deduplicate_all)
 
-    bigeye__experimenter_backend_derived__baseline_clients_daily__v1.set_upstream(
-        experimenter_backend_derived__baseline_clients_daily__v1
-    )
-
-    bigeye__experimenter_backend_derived__baseline_clients_first_seen__v1.set_upstream(
-        experimenter_backend_derived__baseline_clients_first_seen__v1
-    )
-
-    bigeye__experimenter_backend_derived__baseline_clients_last_seen__v1.set_upstream(
-        experimenter_backend_derived__baseline_clients_last_seen__v1
-    )
-
-    bigeye__experimenter_backend_derived__metrics_clients_daily__v1.set_upstream(
-        experimenter_backend_derived__metrics_clients_daily__v1
-    )
-
     bigeye__experimenter_cirrus_derived__baseline_clients_daily__v1.set_upstream(
         experimenter_cirrus_derived__baseline_clients_daily__v1
     )
@@ -7406,10 +7283,6 @@
         burnham_derived__metrics_clients_daily__v1
     )
 
-    checks__fail_experimenter_backend_derived__baseline_clients_last_seen__v1.set_upstream(
-        experimenter_backend_derived__baseline_clients_last_seen__v1
-    )
-
     checks__fail_experimenter_cirrus_derived__baseline_clients_last_seen__v1.set_upstream(
         experimenter_cirrus_derived__baseline_clients_last_seen__v1
     )
@@ -7534,34 +7407,6 @@
         wait_for_copy_deduplicate_all
     )
 
-    experimenter_backend_derived__baseline_clients_daily__v1.set_upstream(
-        bigeye__experimenter_backend_derived__baseline_clients_first_seen__v1
-    )
-
-    experimenter_backend_derived__baseline_clients_daily__v1.set_upstream(
-        wait_for_copy_deduplicate_all
-    )
-
-    experimenter_backend_derived__baseline_clients_first_seen__v1.set_upstream(
-        wait_for_copy_deduplicate_all
-    )
-
-    experimenter_backend_derived__baseline_clients_first_seen__v1.set_upstream(
-        wait_for_telemetry_derived__core_clients_first_seen__v1
-    )
-
-    experimenter_backend_derived__baseline_clients_last_seen__v1.set_upstream(
-        bigeye__experimenter_backend_derived__baseline_clients_daily__v1
-    )
-
-    experimenter_backend_derived__events_stream__v1.set_upstream(
-        wait_for_copy_deduplicate_all
-    )
-
-    experimenter_backend_derived__metrics_clients_daily__v1.set_upstream(
-        wait_for_copy_deduplicate_all
-    )
-
     experimenter_cirrus_derived__baseline_clients_daily__v1.set_upstream(
         bigeye__experimenter_cirrus_derived__baseline_clients_first_seen__v1
     )
@@ -8764,6 +8609,10 @@
 
     relay_backend_derived__events_stream__v1.set_upstream(wait_for_copy_deduplicate_all)
 
+    subscription_platform_backend_cirrus_derived__events_stream__v1.set_upstream(
+        wait_for_copy_deduplicate_all
+    )
+
     subscription_platform_backend_derived__events_stream__v1.set_upstream(
         wait_for_copy_deduplicate_all
     )
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend: baseline
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend: baseline_clients_daily
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend: baseline_clients_first_seen
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend: baseline_clients_last_seen
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend: deletion_request
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend: events_stream
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend: events_unnested
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend: health
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend: metrics
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend: metrics_clients_daily
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived: baseline_clients_daily_v1
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived: baseline_clients_first_seen_v1
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived: baseline_clients_last_seen_v1
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived: event_monitoring_live_v1
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived: events_stream_v1
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend_derived: metrics_clients_daily_v1
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod: subscription_platform_backend_cirrus
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod: subscription_platform_backend_cirrus_derived
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql	2025-11-17 16:07:40.000000000 +0000
@@ -53,7 +53,7 @@
 FROM
   combined
 WHERE
-  DATE(submission_timestamp) >= "2025-11-12"
+  DATE(submission_timestamp) >= "2025-11-17"
 GROUP BY
   submission_date,
   window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_daily_v1/metadata.yaml	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_daily_v1/metadata.yaml	2025-11-17 16:12:33.000000000 +0000
@@ -39,7 +39,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml	2025-11-17 16:12:33.000000000 +0000
@@ -49,7 +49,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml	2025-11-17 16:12:33.000000000 +0000
@@ -48,7 +48,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/event_monitoring_live_v1/materialized_view.sql	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/event_monitoring_live_v1/materialized_view.sql	2025-11-17 16:07:40.000000000 +0000
@@ -53,7 +53,7 @@
 FROM
   combined
 WHERE
-  DATE(submission_timestamp) >= "2025-11-12"
+  DATE(submission_timestamp) >= "2025-11-17"
 GROUP BY
   submission_date,
   window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/metrics_clients_daily_v1/metadata.yaml	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/metrics_clients_daily_v1/metadata.yaml	2025-11-17 16:12:33.000000000 +0000
@@ -34,7 +34,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql	2025-11-17 16:07:41.000000000 +0000
@@ -53,7 +53,7 @@
 FROM
   combined
 WHERE
-  DATE(submission_timestamp) >= "2025-11-12"
+  DATE(submission_timestamp) >= "2025-11-17"
 GROUP BY
   submission_date,
   window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql	2025-11-17 16:07:41.000000000 +0000
@@ -119,7 +119,7 @@
 FROM
   combined
 WHERE
-  DATE(submission_timestamp) >= "2025-11-12"
+  DATE(submission_timestamp) >= "2025-11-17"
 GROUP BY
   submission_date,
   window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_daily_v1/metadata.yaml	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_daily_v1/metadata.yaml	2025-11-17 16:12:34.000000000 +0000
@@ -39,7 +39,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_first_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_first_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_first_seen_v1/metadata.yaml	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_first_seen_v1/metadata.yaml	2025-11-17 16:12:34.000000000 +0000
@@ -49,7 +49,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_last_seen_v1/metadata.yaml	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_last_seen_v1/metadata.yaml	2025-11-17 16:12:34.000000000 +0000
@@ -48,7 +48,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_daily_v1/metadata.yaml	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_daily_v1/metadata.yaml	2025-11-17 16:12:34.000000000 +0000
@@ -34,7 +34,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-11-17 16:12:34.000000000 +0000
@@ -48,7 +48,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql	2025-11-17 16:07:41.000000000 +0000
@@ -53,7 +53,7 @@
 FROM
   combined
 WHERE
-  DATE(submission_timestamp) >= "2025-11-12"
+  DATE(submission_timestamp) >= "2025-11-17"
 GROUP BY
   submission_date,
   window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline/metadata.yaml	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
@@ -1,21 +0,0 @@
-friendly_name: Historical Pings for `experimenter-backend/baseline`
-description: |-
-  A historical view of pings sent for the
-  `experimenter-backend/baseline`
-  document type.
-
-  This view is guaranteed to contain only complete days
-  (per `submission_timestamp`)
-  and to contain only one row per distinct `document_id` within a given date.
-
-  Clustering fields: `normalized_channel`, `sample_id`
-owners: []
-labels: {}
-workgroup_access:
-- role: roles/bigquery.dataViewer
-  members:
-  - workgroup:mozilla-confidential
-references:
-  view.sql:
-  - moz-fx-data-shared-prod.experimenter_backend_stable.baseline_v1
-require_column_descriptions: false
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline/schema.yaml	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline/schema.yaml	1970-01-01 00:00:00.000000000 +0000
@@ -1,508 +0,0 @@
-fields:
-- name: additional_properties
-  type: STRING
-  mode: NULLABLE
-  description: A JSON string containing any payload properties not present in the
-    schema
-- name: client_info
-  type: RECORD
-  mode: NULLABLE
-  fields:
-  - name: android_sdk_version
-    type: STRING
-    mode: NULLABLE
-    description: The optional Android specific SDK version of the software running
-      on this hardware device.
-  - name: app_build
-    type: STRING
-    mode: NULLABLE
-    description: The build identifier generated by the CI system (e.g. "1234/A").
-      For language bindings that provide automatic detection for this value, (e.g.
-      Android/Kotlin), in the unlikely event that the build identifier can not be
-      retrieved from the OS, it is set to "inaccessible". For other language bindings,
-      if the value was not provided through configuration, this metric gets set to
-      `Unknown`.
-  - name: app_channel
-    type: STRING
-    mode: NULLABLE
-    description: The channel the application is being distributed on.
-  - name: app_display_version
-    type: STRING
-    mode: NULLABLE
-    description: The user visible version string (e.g. "1.0.3").  In the unlikely
-      event that the display version can not be retrieved, it is set to "inaccessible".
-  - name: architecture
-    type: STRING
-    mode: NULLABLE
-    description: The architecture of the device, (e.g. "arm", "x86").
-  - name: attribution
-    type: RECORD
-    mode: NULLABLE
-    fields:
-    - name: campaign
-      type: STRING
-      mode: NULLABLE
-      description: The attribution campaign (e.g. 'mozilla-org').
-    - name: content
-      type: STRING
-      mode: NULLABLE
-      description: The attribution content (e.g. 'firefoxview').
-    - name: medium
-      type: STRING
-      mode: NULLABLE
-      description: The attribution medium (e.g. 'organic' for a search engine).
-    - name: source
-      type: STRING
-      mode: NULLABLE
-      description: The attribution source (e.g. 'google-play').
-    - name: term
-      type: STRING
-      mode: NULLABLE
-      description: The attribution term (e.g. 'browser with developer tools for android').
-    - name: ext
-      type: JSON
-      mode: NULLABLE
-  - name: build_date
-    type: STRING
-    mode: NULLABLE
-    description: The date & time the application was built
-  - name: client_id
-    type: STRING
-    mode: NULLABLE
-    description: A UUID uniquely identifying the client.
-  - name: device_manufacturer
-    type: STRING
-    mode: NULLABLE
-    description: The manufacturer of the device the application is running on. Not
-      set if the device manufacturer can't be determined (e.g. on Desktop).
-  - name: device_model
-    type: STRING
-    mode: NULLABLE
-    description: The model of the device the application is running on. On Android,
-      this is Build.MODEL, the user-visible marketing name, like "Pixel 2 XL". Not
-      set if the device model can't be determined (e.g. on Desktop).
-  - name: distribution
-    type: RECORD
-    mode: NULLABLE
-    fields:
-    - name: name
-      type: STRING
-      mode: NULLABLE
-      description: The distribution name (e.g. 'MozillaOnline').
-    - name: ext
-      type: JSON
-      mode: NULLABLE
-  - name: first_run_date
-    type: STRING
-    mode: NULLABLE
-    description: The date of the first run of the application.
-  - name: locale
-    type: STRING
-    mode: NULLABLE
-    description: The locale of the application during initialization (e.g. "es-ES").
-      If the locale can't be determined on the system, the value is ["und"](https://unicode.org/reports/tr35/#Unknown_or_Invalid_Identifiers),
-      to indicate "undetermined".
-  - name: os
-    type: STRING
-    mode: NULLABLE
-    description: 'The name of the operating system. Possible values: Android, iOS,
-      Linux, Darwin, Windows, FreeBSD, NetBSD, OpenBSD, Solaris, unknown'
-  - name: os_version
-    type: STRING
-    mode: NULLABLE
-    description: The user-visible version of the operating system (e.g. "1.2.3").
-      If the version detection fails, this metric gets set to `Unknown`.
-  - name: session_count
-    type: INTEGER
-    mode: NULLABLE
-    description: An optional running counter of the number of sessions for a client.
-  - name: session_id
-    type: STRING
-    mode: NULLABLE
-    description: An optional UUID uniquely identifying the client's current session.
-  - name: telemetry_sdk_build
-    type: STRING
-    mode: NULLABLE
-    description: The version of the Glean SDK
-  - name: windows_build_number
-    type: INTEGER
-    mode: NULLABLE
-    description: The optional Windows build number, reported by Windows (e.g. 22000)
-      and not set for other platforms
-- name: document_id
-  type: STRING
-  mode: NULLABLE
-  description: The document ID specified in the URI when the client sent this message
-- name: events
-  type: RECORD
-  mode: REPEATED
-  fields:
-  - name: category
-    type: STRING
-    mode: NULLABLE
-  - name: extra
-    type: RECORD
-    mode: REPEATED
-    fields:
-    - name: key
-      type: STRING
-      mode: NULLABLE
-    - name: value
-      type: STRING
-      mode: NULLABLE
-  - name: name
-    type: STRING
-    mode: NULLABLE
-  - name: timestamp
-    type: INTEGER
-    mode: NULLABLE
-- name: metadata
-  type: RECORD
-  mode: NULLABLE
-  fields:
-  - name: geo
-    type: RECORD
-    mode: NULLABLE
-    fields:
-    - name: city
-      type: STRING
-      mode: NULLABLE
-    - name: country
-      type: STRING
-      mode: NULLABLE
-      description: An ISO 3166-1 alpha-2 country code
-    - name: db_version
-      type: STRING
-      mode: NULLABLE
-      description: The specific geo database version used for this lookup
-    - name: subdivision1
-      type: STRING
-      mode: NULLABLE
-      description: First major country subdivision, typically a state, province, or
-        county
-    - name: subdivision2
-      type: STRING
-      mode: NULLABLE
-      description: Second major country subdivision; not applicable for most countries
-    description: Results of a geographic lookup based on the client's IP address
-  - name: header
-    type: RECORD
-    mode: NULLABLE
-    fields:
-    - name: date
-      type: STRING
-      mode: NULLABLE
-      description: Date HTTP header
-    - name: dnt
-      type: STRING
-      mode: NULLABLE
-      description: DNT (Do Not Track) HTTP header
-    - name: x_debug_id
-      type: STRING
-      mode: NULLABLE
-      description: X-Debug-Id HTTP header
-    - name: x_foxsec_ip_reputation
-      type: STRING
-      mode: NULLABLE
-      description: X-Foxsec-IP-Reputation header
-    - name: x_lb_tags
-      type: STRING
-      mode: NULLABLE
-      description: X-LB-Tags HTTP header
-    - name: x_pingsender_version
-      type: STRING
-      mode: NULLABLE
-      description: X-PingSender-Version HTTP header
-    - name: x_source_tags
-      type: STRING
-      mode: NULLABLE
-      description: X-Source-Tags HTTP header
-    - name: x_telemetry_agent
-      type: STRING
-      mode: NULLABLE
-      description: X-Telemetry-Agent HTTP header
-    - name: parsed_date
-      type: TIMESTAMP
-      mode: NULLABLE
-    - name: parsed_x_source_tags
-      type: STRING
-      mode: REPEATED
-    - name: parsed_x_lb_tags
-      type: RECORD
-      mode: NULLABLE
-      fields:
-      - name: tls_version
-        type: STRING
-        mode: NULLABLE
-      - name: tls_cipher_hex
-        type: STRING
-        mode: NULLABLE
-    description: Headers included in the client's HTTP request
-  - name: isp
-    type: RECORD
-    mode: NULLABLE
-    fields:
-    - name: db_version
-      type: STRING
-      mode: NULLABLE
-      description: The specific geo ISP database version used for this lookup
-    - name: name
-      type: STRING
-      mode: NULLABLE
-      description: The name of the ISP associated with the client's IP address
-    - name: organization
-      type: STRING
-      mode: NULLABLE
-      description: The name of a specific business entity associated with the client's
-        IP address when available; otherwise the ISP name
-    description: Results of ISP lookup based on the client's IP address
-  - name: user_agent
-    type: RECORD
-    mode: NULLABLE
-    fields:
-    - name: browser
-      type: STRING
-      mode: NULLABLE
-    - name: os
-      type: STRING
-      mode: NULLABLE
-    - name: version
-      type: STRING
-      mode: NULLABLE
-    description: Parsed components of the client's user agent string
-- name: metrics
-  type: RECORD
-  mode: NULLABLE
-  fields:
-  - name: datetime
-    type: RECORD
-    mode: NULLABLE
-    fields:
-    - name: glean_validation_first_run_hour
-      type: TIMESTAMP
-      mode: NULLABLE
-      description: 'The hour of the first run of the application.
-
-        '
-    - name: raw_glean_validation_first_run_hour
-      type: STRING
-      mode: NULLABLE
-  - name: labeled_counter
-    type: RECORD
-    mode: NULLABLE
-    fields:
-    - name: glean_error_invalid_label
-      type: RECORD
-      mode: REPEATED
-      fields:
-      - name: key
-        type: STRING
-        mode: NULLABLE
-      - name: value
-        type: INTEGER
-        mode: NULLABLE
-      description: 'Counts the number of times a metric was set with an invalid label.
-
-        The labels are the `category.name` identifier of the metric.
-
-        '
-    - name: glean_error_invalid_overflow
-      type: RECORD
-      mode: REPEATED
-      fields:
-      - name: key
-        type: STRING
-        mode: NULLABLE
-      - name: value
-        type: INTEGER
-        mode: NULLABLE
-      description: 'Counts the number of times a metric was set a value that overflowed.
-
-        The labels are the `category.name` identifier of the metric.
-
-        '
-    - name: glean_error_invalid_state
-      type: RECORD
-      mode: REPEATED
-      fields:
-      - name: key
-        type: STRING
-        mode: NULLABLE
-      - name: value
-        type: INTEGER
-        mode: NULLABLE
-      description: 'Counts the number of times a timing metric was used incorrectly.
-
-        The labels are the `category.name` identifier of the metric.
-
-        '
-    - name: glean_error_invalid_value
-      type: RECORD
-      mode: REPEATED
-      fields:
-      - name: key
-        type: STRING
-        mode: NULLABLE
-      - name: value
-        type: INTEGER
-        mode: NULLABLE
-      description: 'Counts the number of times a metric was set to an invalid value.
-
-        The labels are the `category.name` identifier of the metric.
-
-        '
-    - name: glean_validation_pings_submitted
-      type: RECORD
-      mode: REPEATED
-      fields:
-      - name: key
-        type: STRING
-        mode: NULLABLE
-      - name: value
-        type: INTEGER
-        mode: NULLABLE
-      description: "A count of the built-in pings submitted, by ping type.\n\nThis\
-        \ metric appears in both the metrics and baseline pings.\n\n- On the metrics\
-        \ ping, the counts include the number of pings sent since\n  the last metrics\
-        \ ping (including the last metrics ping)\n- On the baseline ping, the counts\
-        \ include the number of pings send since\n  the last baseline ping (including\
-        \ the last baseline ping)\n\nNote: Previously this also recorded the number\
-        \ of submitted custom pings.\nNow it only records counts for the Glean built-in\
-        \ pings.\n"
-  - name: string
-    type: RECORD
-    mode: NULLABLE
-    fields:
-    - name: glean_baseline_locale
-      type: STRING
-      mode: NULLABLE
-      description: 'The locale of the application during initialization (e.g. "es-ES").
-
-        If the locale can''t be determined on the system, the value is
-
-        ["und"](https://unicode.org/reports/tr35/#Unknown_or_Invalid_Identifiers),
-
-        to indicate "undetermined".
-
-        '
-    - name: glean_client_annotation_experimentation_id
-      type: STRING
-      mode: NULLABLE
-      description: 'An experimentation identifier derived and provided by the application
-
-        for the purpose of experimentation enrollment.
-
-        '
-  - name: string_list
-    type: RECORD
-    mode: NULLABLE
-    fields:
-    - name: glean_ping_uploader_capabilities
-      type: STRING
-      mode: REPEATED
-  - name: timespan
-    type: RECORD
-    mode: NULLABLE
-    fields:
-    - name: glean_baseline_duration
-      type: RECORD
-      mode: NULLABLE
-      fields:
-      - name: time_unit
-        type: STRING
-        mode: NULLABLE
-      - name: value
-        type: INTEGER
-        mode: NULLABLE
-      description: 'The duration of the last foreground session.
-
-        '
-- name: normalized_app_name
-  type: STRING
-  mode: NULLABLE
-  description: Set to "Other" if this message contained an unrecognized app name
-- name: normalized_channel
-  type: STRING
-  mode: NULLABLE
-  description: Set to "Other" if this message contained an unrecognized channel name
-- name: normalized_country_code
-  type: STRING
-  mode: NULLABLE
-  description: An ISO 3166-1 alpha-2 country code
-- name: normalized_os
-  type: STRING
-  mode: NULLABLE
-  description: Set to "Other" if this message contained an unrecognized OS name
-- name: normalized_os_version
-  type: STRING
-  mode: NULLABLE
-- name: ping_info
-  type: RECORD
-  mode: NULLABLE
-  fields:
-  - name: end_time
-    type: STRING
-    mode: NULLABLE
-  - name: experiments
-    type: RECORD
-    mode: REPEATED
-    fields:
-    - name: key
-      type: STRING
-      mode: NULLABLE
-    - name: value
-      type: RECORD
-      mode: NULLABLE
-      fields:
-      - name: branch
-        type: STRING
-        mode: NULLABLE
-      - name: extra
-        type: RECORD
-        mode: NULLABLE
-        fields:
-        - name: enrollment_id
-          type: STRING
-          mode: NULLABLE
-        - name: type
-          type: STRING
-          mode: NULLABLE
-  - name: ping_type
-    type: STRING
-    mode: NULLABLE
-  - name: reason
-    type: STRING
-    mode: NULLABLE
-  - name: seq
-    type: INTEGER
-    mode: NULLABLE
-  - name: start_time
-    type: STRING
-    mode: NULLABLE
-  - name: parsed_start_time
-    type: TIMESTAMP
-    mode: NULLABLE
-  - name: parsed_end_time
-    type: TIMESTAMP
-    mode: NULLABLE
-- name: sample_id
-  type: INTEGER
-  mode: NULLABLE
-  description: Hashed version of client_id (if present) useful for partitioning; ranges
-    from 0 to 99
-- name: submission_timestamp
-  type: TIMESTAMP
-  mode: NULLABLE
-  description: Time when the ingestion edge server accepted this message
-- name: app_version_major
-  type: NUMERIC
-  mode: NULLABLE
-- name: app_version_minor
-  type: NUMERIC
-  mode: NULLABLE
-- name: app_version_patch
-  type: NUMERIC
-  mode: NULLABLE
-- name: is_bot_generated
-  type: BOOLEAN
-  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline/view.sql	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline/view.sql	1970-01-01 00:00:00.000000000 +0000
@@ -1,31 +0,0 @@
--- Generated via ./bqetl generate stable_views
-CREATE OR REPLACE VIEW
-  `moz-fx-data-shared-prod.experimenter_backend.baseline`
-AS
-SELECT
-  * REPLACE (
-    mozfun.norm.metadata(metadata) AS metadata,
-    mozfun.norm.glean_ping_info(ping_info) AS ping_info,
-    (
-      SELECT AS STRUCT
-        metrics.* REPLACE (
-          STRUCT(
-            mozfun.glean.parse_datetime(
-              metrics.datetime.glean_validation_first_run_hour
-            ) AS glean_validation_first_run_hour,
-            metrics.datetime.glean_validation_first_run_hour AS raw_glean_validation_first_run_hour
-          ) AS datetime
-        )
-    ) AS metrics,
-    mozfun.norm.glean_client_info_attribution(
-      mozfun.norm.glean_baseline_client_info(client_info, metrics),
-      CAST(NULL AS JSON),
-      CAST(NULL AS JSON)
-    ) AS client_info
-  ),
-  mozfun.norm.extract_version(client_info.app_display_version, 'major') AS app_version_major,
-  mozfun.norm.extract_version(client_info.app_display_version, 'minor') AS app_version_minor,
-  mozfun.norm.extract_version(client_info.app_display_version, 'patch') AS app_version_patch,
-  LOWER(IFNULL(metadata.isp.name, "")) = "browserstack" AS is_bot_generated,
-FROM
-  `moz-fx-data-shared-prod.experimenter_backend_stable.baseline_v1`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline_clients_daily/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline_clients_daily/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline_clients_daily/metadata.yaml	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline_clients_daily/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
@@ -1,19 +0,0 @@
-friendly_name: Baseline Clients Daily
-description: |-
-  A daily aggregate of baseline pings per `client_id`.
-
-  Clustering fields: `normalized_channel`, `sample_id`
-
-  See also: `baseline_clients_last_seen`
-owners:
-- [email protected]
-labels:
-  owner1: ascholtz
-workgroup_access:
-- role: roles/bigquery.dataViewer
-  members:
-  - workgroup:mozilla-confidential
-references:
-  view.sql:
-  - moz-fx-data-shared-prod.experimenter_backend_derived.baseline_clients_daily_v1
-require_column_descriptions: false
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline_clients_daily/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline_clients_daily/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline_clients_daily/view.sql	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline_clients_daily/view.sql	1970-01-01 00:00:00.000000000 +0000
@@ -1,8 +0,0 @@
--- Generated via bigquery_etl.glean_usage
-CREATE OR REPLACE VIEW
-  `moz-fx-data-shared-prod.experimenter_backend.baseline_clients_daily`
-AS
-SELECT
-  *
-FROM
-  `moz-fx-data-shared-prod.experimenter_backend_derived.baseline_clients_daily_v1`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline_clients_first_seen/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline_clients_first_seen/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline_clients_first_seen/metadata.yaml	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline_clients_first_seen/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
@@ -1,23 +0,0 @@
-friendly_name: Baseline Clients First Seen
-description: |-
-  Captures the earliest server date that we observe a particular client in the
-  baseline table. A client may have a `first_seen_date` that is earlier than the
-  epoch of the baseline table; this is the result of a migration from a legacy
-  data collection system. These clients are therefore not captured in this
-  table.
-
-  Clustering fields: `sample_id`
-
-  See also: `baseline_clients_daily` and `baseline_clients_last_seen`.
-owners:
-- [email protected]
-labels:
-  owner1: ascholtz
-workgroup_access:
-- role: roles/bigquery.dataViewer
-  members:
-  - workgroup:mozilla-confidential
-references:
-  view.sql:
-  - moz-fx-data-shared-prod.experimenter_backend_derived.baseline_clients_daily_v1
-require_column_descriptions: false
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline_clients_first_seen/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline_clients_first_seen/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline_clients_first_seen/view.sql	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline_clients_first_seen/view.sql	1970-01-01 00:00:00.000000000 +0000
@@ -1,10 +0,0 @@
--- Generated via bigquery_etl.glean_usage
-CREATE OR REPLACE VIEW
-  `moz-fx-data-shared-prod.experimenter_backend.baseline_clients_first_seen`
-AS
-SELECT
-  *
-FROM
-  `moz-fx-data-shared-prod.experimenter_backend_derived.baseline_clients_daily_v1`
-WHERE
-  is_new_profile
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline_clients_last_seen/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline_clients_last_seen/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline_clients_last_seen/metadata.yaml	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline_clients_last_seen/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
@@ -1,20 +0,0 @@
-friendly_name: Baseline Clients Last Seen
-description: |-
-  Captures activity history of each client in 28-day
-  windows for each submission date based on baseline pings.
-
-  Clustering fields: `normalized_channel`, `sample_id`
-
-  See also: `baseline_clients_daily`
-owners:
-- [email protected]
-labels:
-  owner1: ascholtz
-workgroup_access:
-- role: roles/bigquery.dataViewer
-  members:
-  - workgroup:mozilla-confidential
-references:
-  view.sql:
-  - moz-fx-data-shared-prod.experimenter_backend_derived.baseline_clients_last_seen_v1
-require_column_descriptions: false
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline_clients_last_seen/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline_clients_last_seen/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline_clients_last_seen/view.sql	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/baseline_clients_last_seen/view.sql	1970-01-01 00:00:00.000000000 +0000
@@ -1,22 +0,0 @@
--- Generated via bigquery_etl.glean_usage
-CREATE OR REPLACE VIEW
-  `moz-fx-data-shared-prod.experimenter_backend.baseline_clients_last_seen`
-AS
-SELECT
-  `moz-fx-data-shared-prod`.udf.pos_of_trailing_set_bit(days_seen_bits) AS days_since_seen,
-  `moz-fx-data-shared-prod`.udf.pos_of_trailing_set_bit(days_active_bits) AS days_since_active,
-  `moz-fx-data-shared-prod`.udf.pos_of_trailing_set_bit(
-    days_created_profile_bits
-  ) AS days_since_created_profile,
-  `moz-fx-data-shared-prod`.udf.pos_of_trailing_set_bit(
-    days_seen_session_start_bits
-  ) AS days_since_seen_session_start,
-  `moz-fx-data-shared-prod`.udf.pos_of_trailing_set_bit(
-    days_seen_session_end_bits
-  ) AS days_since_seen_session_end,
-  `moz-fx-data-shared-prod`.udf.pos_of_trailing_set_bit(
-    days_visited_1_uri_bits
-  ) AS days_since_visited_1_uri,
-  *
-FROM
-  `moz-fx-data-shared-prod.experimenter_backend_derived.baseline_clients_last_seen_v1`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/data_collection_opt_out/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/data_collection_opt_out/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/data_collection_opt_out/schema.yaml	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/data_collection_opt_out/schema.yaml	2025-11-17 16:04:12.000000000 +0000
@@ -274,93 +274,14 @@
   type: RECORD
   mode: NULLABLE
   fields:
-  - name: labeled_counter
-    type: RECORD
-    mode: NULLABLE
-    fields:
-    - name: glean_error_invalid_label
-      type: RECORD
-      mode: REPEATED
-      fields:
-      - name: key
-        type: STRING
-        mode: NULLABLE
-      - name: value
-        type: INTEGER
-        mode: NULLABLE
-      description: 'Counts the number of times a metric was set with an invalid label.
-
-        The labels are the `category.name` identifier of the metric.
-
-        '
-    - name: glean_error_invalid_overflow
-      type: RECORD
-      mode: REPEATED
-      fields:
-      - name: key
-        type: STRING
-        mode: NULLABLE
-      - name: value
-        type: INTEGER
-        mode: NULLABLE
-      description: 'Counts the number of times a metric was set a value that overflowed.
-
-        The labels are the `category.name` identifier of the metric.
-
-        '
-    - name: glean_error_invalid_state
-      type: RECORD
-      mode: REPEATED
-      fields:
-      - name: key
-        type: STRING
-        mode: NULLABLE
-      - name: value
-        type: INTEGER
-        mode: NULLABLE
-      description: 'Counts the number of times a timing metric was used incorrectly.
-
-        The labels are the `category.name` identifier of the metric.
-
-        '
-    - name: glean_error_invalid_value
-      type: RECORD
-      mode: REPEATED
-      fields:
-      - name: key
-        type: STRING
-        mode: NULLABLE
-      - name: value
-        type: INTEGER
-        mode: NULLABLE
-      description: 'Counts the number of times a metric was set to an invalid value.
-
-        The labels are the `category.name` identifier of the metric.
-
-        '
   - name: string
     type: RECORD
     mode: NULLABLE
     fields:
-    - name: glean_client_annotation_experimentation_id
-      type: STRING
-      mode: NULLABLE
-      description: 'An experimentation identifier derived and provided by the application
-
-        for the purpose of experimentation enrollment.
-
-        '
     - name: nimbus_nimbus_user_id
       type: STRING
       mode: NULLABLE
       description: The id of the user sent to Cirrus to determine enrollments.
-  - name: string_list
-    type: RECORD
-    mode: NULLABLE
-    fields:
-    - name: glean_ping_uploader_capabilities
-      type: STRING
-      mode: REPEATED
 - name: normalized_app_name
   type: STRING
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/deletion_request/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/deletion_request/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/deletion_request/metadata.yaml	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/deletion_request/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
@@ -1,21 +0,0 @@
-friendly_name: Historical Pings for `experimenter-backend/deletion-request`
-description: |-
-  A historical view of pings sent for the
-  `experimenter-backend/deletion-request`
-  document type.
-
-  This view is guaranteed to contain only complete days
-  (per `submission_timestamp`)
-  and to contain only one row per distinct `document_id` within a given date.
-
-  Clustering fields: `normalized_channel`, `sample_id`
-owners: []
-labels: {}
-workgroup_access:
-- role: roles/bigquery.dataViewer
-  members:
-  - workgroup:mozilla-confidential
-references:
-  view.sql:
-  - moz-fx-data-shared-prod.experimenter_backend_stable.deletion_request_v1
-require_column_descriptions: false
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/deletion_request/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/deletion_request/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/deletion_request/schema.yaml	2025-11-17 16:06:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_backend/deletion_request/schema.yaml	1970-01-01 00:00:00.000000000 +0000
@@ -1,448 +0,0 @@
-fields:
-- name: additional_properties
-  type: STRING
-  mode: NULLABLE
-  description: A JSON string containing any payload properties not present in the
-    schema
-- name: client_info
-  type: RECORD
-  mode: NULLABLE
-  fields:
-  - name: android_sdk_version
-    type: STRING
-    mode: NULLABLE
-    description: The optional Android specific SDK version of the software running
-      on this hardware device.
-  - name: app_build
-    type: STRING
-    mode: NULLABLE
-    description: The build identifier generated by the CI system (e.g. "1234/A").
-      For language bindings that provide automatic detection for this value, (e.g.
-      Android/Kotlin), in the unlikely event that the build identifier can not be
-      retrieved from the OS, it is set to "inaccessible". For other language bindings,
-      if the value was not provided through configuration, this metric gets set to
-      `Unknown`.
-  - name: app_channel
-    type: STRING
-    mode: NULLABLE
-    description: The channel the application is being distributed on.
-  - name: app_display_version
-    type: STRING
-    mode: NULLABLE
-    description: The user visible version string (e.g. "1.0.3").  In the unlikely
-      event that the display version can not be retrieved, it is set to "inaccessible".
-  - name: architecture
-    type: STRING
-    mode: NULLABLE
-    description: The architecture of the device, (e.g. "arm", "x86").
-  - name: attribution
-    type: RECORD
-    mode: NULLABLE
-    fields:
-    - name: campaign
-      type: STRING
-      mode: NULLABLE
-      description: The attribution campaign (e.g. 'mozilla-org').
-    - name: content
-      type: STRING
-      mode: NULLABLE
-      description: The attribution content (e.g. 'firefoxview').
-    - name: medium
-      type: STRING
-      mode: NULLABLE
-      description: The attribution medium (e.g. 'organic' for a search engine).
-    - name: source
-      type: STRING
-      mode: NULLABLE
-      description: The attribution source (e.g. 'google-play').
-    - name: term
-      type: STRING
-      mode: NULLABLE
-      description: The attribution term (e.g. 'browser with developer tools for android').
-    - name: ext
-      type: JSON
-      mode: NULLABLE
-  - name: build_date
-    type: STRING
-    mode: NULLABLE
-    description: The date & time the application was built
-  - name: client_id
-    type: STRING
-    mode: NULLABLE
-    description: A UUID uniquely identifying the client.
-  - name: device_manufacturer
-    type: STRING
-    mode: NULLABLE
-    description: The manufacturer of the device the application is running on. Not
-      set if the device manufacturer can't be determined (e.g. on Desktop).
-  - name: device_model
-    type: STRING
-    mode: NULLABLE
-    description: The model of the device the application is running on. On Android,
-      this is Build.MODEL, the user-visible marketing name, like "Pixel 2 XL". Not
-      set if the device model can't be determined (e.g. on Desktop).
-  - name: distribution
-    type: RECORD
-    mode: NULLABLE
-    fields:
-    - name: name
-      type: STRING
-      mode: NULLABLE
-      description: The distribution name (e.g. 'MozillaOnline').
-    - name: ext
-      type: JSON
-      mode: NULLABLE
-  - name: first_run_date
-    type: STRING
-    mode: NULLABLE
-    description: The date of the first run of the application.
-  - name: locale
-    type: STRING
-    mode: NULLABLE
-    description: The locale of the application during initialization (e.g. "es-ES").
-      If the locale can't be determined on the system, the value is ["und"](https://unicode.org/reports/tr35/#Unknown_or_Invalid_Identifiers),
-      to indicate "undetermined".
-  - name: os
-    type: STRING
-    mode: NULLABLE
-    description: 'The name of the operating system. Possible values: Android, iOS,
-      Linux, Darwin, Windows, FreeBSD, NetBSD, OpenBSD, Solaris, unknown'
-  - name: os_version
-    type: STRING
-    mode: NULLABLE
-    description: The user-visible version of the operating system (e.g. "1.2.3").
-      If the version detection fails, this metric gets set to `Unknown`.
-  - name: session_count
-    type: INTEGER
-    mode: NULLABLE
-    description: An optional running counter of the number of sessions for a client.
-  - name: session_id
-    type: STRING
-    mode: NULLABLE
-    description: An optional UUID uniquely identifying the client's current session.
-  - name: telemetry_sdk_build
-    type: STRING
-    mode: NULLABLE
-    description: The version of the Glean SDK
-  - name: windows_build_number
-    type: INTEGER
-    mode: NULLABLE
-    description: The optional Windows build number, reported by Windows (e.g. 22000)
-      and not set for other platforms
-- name: document_id
-  type: STRING
-  mode: NULLABLE
-  description: The document ID specified in the URI when the client sent this message
-- name: events
-  type: RECORD
-  mode: REPEATED
-  fields:
-  - name: category
-    type: STRING
-    mode: NULLABLE
-  - name: extra
-    type: RECORD
-    mode: REPEATED
-    fields:
-    - name: key
-      type: STRING
-      mode: NULLABLE
-    - name: value
-      type: STRING
-      mode: NULLABLE
-  - name: name
-    type: STRING
-    mode: NULLABLE
-  - name: timestamp
-    type: INTEGER
-    mode: NULLABLE
-- name: metadata
-  type: RECORD
-  mode: NULLABLE
-  fields:
-  - name: geo
-    type: RECORD
-    mode: NULLABLE
-    fields:
-    - name: city
-      type: STRING
-      mode: NULLABLE
-    - name: country
-      type: STRING
-      mode: NULLABLE
-      description: An ISO 3166-1 alpha-2 country code
-    - name: db_version
-      type: STRING
-      mode: NULLABLE
-      description: The specific geo database version used for this lookup
-    - name: subdivision1
-      type: STRING
-      mode: NULLABLE
-      description: First major country subdivision, typically a state, province, or
-        county
-    - name: subdivision2
-      type: STRING
-      mode: NULLABLE
-      description: Second major country subdivision; not applicable for most countries
-    description: Results of a geographic lookup based on the client's IP address
-  - name: header
-    type: RECORD
-    mode: NULLABLE
-    fields:
-    - name: date
-      type: STRING
-      mode: NULLABLE
-      description: Date HTTP header
-    - name: dnt
-      type: STRING
-      mode: NULLABLE
-      description: DNT (Do Not Track) HTTP header
-    - name: x_debug_id
-      type: STRING
-      mode: NULLABLE
-      description: X-Debug-Id HTTP header
-    - name: x_foxsec_ip_reputation
-      type: STRING
-      mode: NULLABLE
-      description: X-Foxsec-IP-Reputation header
-    - name: x_lb_tags
-      type: STRING
-      mode: NULLABLE
-      description: X-LB-Tags HTTP header
-    - name: x_pingsender_version
-      type: STRING
-      mode: NULLABLE
-      description: X-PingSender-Version HTTP header
-    - name: x_source_tags
-      type: STRING
-      mode: NULLABLE
-      description: X-Source-Tags HTTP header
-    - name: x_telemetry_agent
-      type: STRING
-      mode: NULLABLE
-      description: X-Telemetry-Agent HTTP header
-    - name: parsed_date
-      type: TIMESTAMP
-      mode: NULLABLE
-    - name: parsed_x_source_tags
-      type: STRING
-      mode: REPEATED
-    - name: parsed_x_lb_tags
-      type: RECORD
-      mode: NULLABLE
-      fields:
-      - name: tls_version
-        type: STRING
-        mode: NULLABLE
-      - name: tls_cipher_hex
-        type: STRING
-        mode: NULLABLE
-    description: Headers included in the client's HTTP request
-  - name: isp
-    type: RECORD
-    mode: NULLABLE
-    fields:
-    - name: db_version
-      type: STRING
-      mode: NULLABLE
-      description: The specific geo ISP database version used for this lookup
-    - name: name
-      type: STRING
-      mode: NULLABLE
-      description: The name of the ISP associated with the client's IP address
-    - name: organization
-      type: STRING
-      mode: NULLABLE
-      description: The name of a specific business entity associated with the client's
-        IP address when available; otherwise the ISP name
-    description: Results of ISP lookup based on the client's IP address
-  - name: user_agent
-    type: RECORD
-    mode: NULLABLE
-    fields:
-    - name: browser
-      type: STRING
-      mode: NULLABLE
-    - name: os
-      type: STRING
-      mode: NULLABLE
-    - name: version
-      type: STRING
-      mode: NULLABLE
-    description: Parsed components of the client's user agent string
-- name: metrics
-  type: RECORD
-  mode: NULLABLE
-  fields:
-  - name: labeled_counter
-    type: RECORD
-    mode: NULLABLE
-    fields:
-    - name: glean_error_invalid_label
-      type: RECORD
-      mode: REPEATED
-      fields:
-      - name: key
-        type: STRING
-        mode: NULLABLE
-      - name: value
-        type: INTEGER
-        mode: NULLABLE
-      description: 'Counts the number of times a metric was set with an invalid label.
-
-        The labels are the `category.name` identifier of the metric.
-
-        '
-    - name: glean_error_invalid_overflow
-      t

⚠️ Only part of the diff is displayed.

Link to full diff

[scholtzan]

The SQL generator could be renamed to something like stable_table_monitoring to make it clearer that it is generating monitoring related configs

[scholtzan]

the friendly name should match that of the related stable table

[scholtzan]

Two things that may need to be taken into account:

1. When bqetl metadata publish '*' is running as part of the deploys, we want to skip publishing metadata to the stable tables. Airflow won't have permissions to write metadata to them anyways. This might require adding some config to bqetl_project.yaml to allow configuring that stable tables should be skipped.
2. Something to manually check: for stage and prod deploys having the generated bigeye and metadata files shouldn't trigger deploys for these tables, or schema generations. The stable tables get deployed through a different mechanism (pipeline schema deploys). This step might already not be happening since there are no query files, but better to test.