Article

    Audience Snapshotting

    8 min read
    Last updated 3 months ago

    Learn how Snapshotting keeps a history of your audiences with every export

    Do you know how GrowthLoop helps keep track of your audience changes over time? It’s all thanks to Snapshotting.

    Every time you export an audience, GrowthLoop writes a set of snapshot tables to your data warehouse creating a complete history you can query, compare, and use to power insights like Audience Overlap and Audience Evaluation. With Snapshotting, you can:

    1. Track when users are added or removed from an audience
    2. Compare audience composition between exports
    3. Unlock GrowthLoop features like Audience Overlap and Audience Evaluation

    Here are a few key things to know about how it works and what’s required to use it effectively.

    • Export triggers snapshotting: When you export an audience, GrowthLoop writes four snapshot tables to the system schema defined in your source connection configuration (commonly named flywheel_system, but this may vary based on your setup):
      • Metadata: General info about the snapshot run
      • Snapshot History: Full audience state at the time of export
      • Delta History: Adds/removes since the last export
      • Unloaded Deltas: Log of runs written to cloud storage
    • Full or Incremental updates: For any destination, you can choose whether to:
      • Export the full audience every time
      • Export only incremental changes (adds/removes) since the last export
    • Pre-snapshot checks: Before writing to your warehouse, GrowthLoop runs data integrity checks to ensure there are no duplicates or null values in your primary key.
    • Exports only: Snapshots are only created when an audience is exported. If you save or update an audience without exporting it, no snapshot tables will be written. This helps control data warehousing costs by avoiding unnecessary storage.
    • Warehouse requirements: Snapshotting requires that your data warehouse connection is active and properly configured, with a system schema defined and GrowthLoop having read/write access. This applies to all supported data warehouses.

    Now that you know the key details about how Snapshotting works, let’s take a closer look at each of the snapshot tables and what data they contain.

    When Snapshotting runs, it doesn’t just create one table - it gives you four, each with its own job. Think of them as different “angles” on your audience’s history: one for the big picture, one for the fine details, one for just the changes, and one for where things are stored. Let's dive into each one:

    A quick reference sheet for the snapshot run. It tells you when it happened, what audience it was for, the primary keys used, and other high-level details.

    Naming Pattern:

    snapshotting_audience_id_metadata
    

    Example:

    snapshotting_audience_535_metadata
    

    Schema:

    Field NameData TypeDescription
    _created_atTimestampTimestamp when the record was created
    _external_run_idStringUnique identifier generated by the client initiating the snapshot
    _snapshot_run_idStringUnique identifier for the snapshot run
    _key_fieldsStringPrimary key field
    _key_fields_hashedStringHashed primary key field
    _export_idsStringThe ID for the request that triggered the node evaluation
    _control_percentageFloatPercentage of the audience assigned to the control group
    _base_query_json_hashedStringHash of the audience’s base query in JSON format

    A full record of the audience at the moment it was exported. Every member, every attribute - this is the complete “frozen in time” view.

    Naming Pattern:

    snapshotting_audience_id_snapshot_history
    

    Example:

    snapshotting_audience_535_snapshot_history
    

    Schema:

    Field NameData TypeDescription
    _created_atTimestampTimestamp for record creation
    _external_run_idStringA unique identifier generated by the client that initiated the snapshot request
    _snapshot_run_idStringA unique identifier for each snapshot run
    _snap_hashStringInternal hash of the snapshot
    _row_hashStringInternal hash of the row
    _json_rowStringJSON representation of the row
    _is_controlBooleanWhether customer is assigned to control group
    customer_idStringCustomer ID
    audience_nameStringName of the audience
    audience_idIntegerID of the audience
    flywheel_export_run_idStringGrowthLoop export run ID
    client_customer_idStringClient-provided customer ID
    external_idStringExternal ID
    customer_attributeStringCustom attribute value
    _export_idStringExport ID

    The “what’s new” file. It only shows the changes since the last export like who was added, who was removed - so you can focus on just the updates without scanning the whole audience.

    Naming Pattern:

    snapshotting_audience_id_delta_history
    

    Example:

    snapshotting_audience_535_delta_history
    

    Schema:

    Field NameData TypeDescription
    _created_atTimestampTimestamp for record creation
    _external_run_idStringA unique identifier generated by the client that initiated the snapshot request
    _snapshot_run_idStringA unique identifier for each snapshot run
    _previous_external_run_idStringExternal run ID for the previous snapshot
    _snap_hashStringInternal hash of the snapshot
    _json_rowStringJSON representation of the row
    _is_controlBooleanWhether customer is assigned to control group
    _export_idStringExport ID
    customer_idStringCustomer ID
    audience_nameStringName of the audience
    audience_idIntegerID of the audience
    flywheel_export_run_idStringGrowthLoop export run ID
    client_customer_idStringClient-provided customer ID
    external_idStringExternal ID
    customer_attributeStringCustom attribute value
    _export_idStringExport ID

    A log of export runs that were written out to your cloud storage. If you ever need to track exactly where a snapshot landed, this is where to look.

    Naming Pattern:

    snapshotting_audience_[AUDIENCE_ID]_unloaded_deltas
    

    Example:

    snapshotting_audience_535_unloaded_deltas
    

    Schema:

    Field NameData TypeDescription
    _created_atTimestampTimestamp when the record was created
    _external_run_idStringUnique identifier generated by the client initiating the snapshot
    _snapshot_run_idStringUnique identifier for the snapshot run
    _cloud_storage_uriStringLocation in cloud storage where the data was written
    _export_idStringExport ID

    You’ve now got the full picture of what each snapshot table does and how they work together. The best way to get comfortable with them? Start exploring.

    • Kick off an export and watch the snapshot tables appear in your flywheel_system schema.
    • Take a peek at the data in each table to see how the Metadata, History, and Delta views connect.
    • Try a few quick queries to spot changes, track growth, or confirm that exports ran as expected.

    Over time, your snapshot history becomes a powerful record, one you can rely on to answer “what changed, when, and why” for any audience.

    📘

    Have questions or feedback?

    We’d love to hear from you! Reach out to us at [email protected] and we’ll be in touch shortly to help resolve the matter!

    Updated 2 months ago