Rolling window data backup within Domo

Background

Advertising is changing fast, and so is the data that comes with it. Our agency combines data from services like Facebook Ads, Google’s Campaign Manager 360, Google Ads, Bing Ads, Snapchat, and others so we can track and optimize advertising campaigns across channels together in one place – Domo. However, some of the services listed above delete data from their system after 24 months. Additionally, we are interested in a rolling window of data, as advertising data can change up to 90 days (or longer!) after an action has occurred, as attribution modeling updates or fraudulent clicks are removed.

When pulling data, we can easily change date ranges in pre-built Domo connectors and use the append functionality to capture historic data. We can then create a safe haven within Domo for our data and know that it won’t get deleted. However, while Domo has 99% of the connectors we need, sometimes we need to work with the Domo API to create a custom solution.

Problem Statement

As an example, let’s say I’ve developed a solution that pulls data from 91 days ago until yesterday into Domo. I want to save the historic data after that 90-day rolling window (the 91^st day), but I don’t want to spend time redeploying my solution with a different date range in the pull. This is where Domo comes back in. 

Solution Statement

We can leverage the DataSet Copy Unload Connector to create a loop within Domo that backs up data. The DataSet Copy Unload Connector is easy to use. All you need to do are setup Domo API credentials, then specify the Dataset ID you want to copy, and the Domo instance you’re grabbing the data from. 

Demo

First, we set up an ETL that uses a filter-formula to grab the information from 91 days ago from our custom connection. Then, we can use the DataSet Copy Unload Connector with the dataset ID from the dataset produced by the ETL, set the update mode to append, and set it to run on a schedule to update before the original connection runs again. And just like that, you’ve got your historic data from a rolling window of data safe in Domo! I hope this was helpful and introduced you to something new in Domo!

The post Rolling window data backup within Domo first appeared on Blog.