Creating Operational Data Export Schedules

When you have created some Export Destinations using S3 buckets in your AWS account or using buckets in your Google Cloud Storage account, you can set up Export Schedules to perform operational data exports into those destinations from your m3ter Organization.

This topic explains how to work with and create Export Schedules for exporting operational data from your Organization at regular intervals:

Note: Export jobs for Export schedules. When you have created a data Export schedule, an export job is created for each time the data export is due to run under the schedule. You can review and check the status of data export jobs on the Exports job page - see Reviewing Export Jobs.

Working with Operational Data Export Schedules

This section explains how the Data Export service will compile and run operational data exports according to the various options available to you when setting up your operational data Export Schedules:

Data Export Service - Timezone Used

The Data Export service uses UTC and this is not affected by the Timezone you've set for your Organization.

When Export Schedules are Run

When you've completed the configuration for an Export Schedule and select to create it:

An Export job is scheduled to run the data export immediately with the Data Export service.
Export jobs for the next and subsequent in the series of exports are then scheduled to run with the Data Export Service according to the minutes, hours, or days frequency you've configured for the Schedule.
For example, if you select to create an Export schedule at 10:00 a.m. on June 1st, 2024 UTC and the Export Schedule is configured for a frequency of every 5 days, then: an Export job is scheduled to run with the Data Export Service immediately; an Export job for the next data export will then be scheduled to run at 10:00 a.m. on June 6th 2024 UTC; and so on.

Export File Format and Composition

The range of operational data included in an export depends on the operational entities you select for the export.

File Format, Composition, and Entity Schema

File Format

The available data export file formats are .CSV or JSONL.

File Composition

Generally, the fields returned with entities by Data Exports are the same as are returned by the API when retrieving an entity, plus some additional fields only relevant for exports.

Data Export Entity Schema

You can review the details of export schema for each entity available for operational data export - see Data Export Entity Schema.

Operational Entities

You can select to export the operational data in your m3ter Organization for the following entities:

Accounts
Account Plans
Aggregations
Balances
Balance Transactions
Bills
Bill line items
Prepayments/Commitments
Compound Aggregations
Contracts
Meters
Plans
Plan Groups
Plan Group Links
Plan Templates
Pricing
Pricing Bands
Products

A file is compiled for each of the entity types you select.

Export Size Limit

Typically Operational Data exported files are relatively small, but if they do exceed 6GB the exported data is split across multiple files.

Creating Operational Data Export Schedules

You can quickly set up scheduled exports of operational data such as Accounts, Bills, and Balances. Scheduled exports are incremental and designed to be easily integrated into downstream data tools.

To create an Operational Data Export Schedule:

1. Select Data exports:

The Data export page opens and lists any existing Export schedules.

2. Select Create Export schedule. The Create page opens.

3. Enter the Export schedule details:

Name. Descriptive name for the schedule.
Destinations. Use the drop-down to select the Export destinations to be used each time the data export runs for the schedule:
- Select the Export Destination(s) you want the export to be delivered to.
- If No options shows in the drop-down, you must first create at least one Destination before you can set up an Export Schedule - see Creating and Managing Export Destinations.
- If no Destination is selected, the export will be available to download locally. See Downloading Data Export Files Locally for more details.
Export file format. Select the format you require for the usage data export files:
- CSV. (Default).
- JSONL.
Frequency. Define the frequency for the schedule in Hours, Days, or Minutes. Limits are imposed on the lowest and highest frequency, and errors will show if you try to set a frequency outside of these limits:
- Lowest frequency is every 3 days.
- Highest frequency is every 15 minutes.
Source type. Select the Operational data option. The page adjusts to allow you to select the operational entities whose data you want the export to contain.

4. Under Operational data entities, select the plus icon and use the pop-up to select the entities whose data you want to include in the export:

5. Select Add. the selected entities are added to Export schedule details:

In this example, we've selected to include the operational data for Accounts, Account Plans, and Prepayments/Commitments in the data exports ran every five days for the Export schedule.

6. When you have entered all details, select Create export schedule. You are taken back to the Data export page where the new schedule is listed on the Export schedules panel: