Reviewing and Resolving Ingest Events
If a usage data measurement you submit to the platform fails to ingest properly, then an ingest validation failure system Event is generated for the ingest error that has occurred:
ingest.validation.failure
In the Usage section of the Console on the Ingest Events page, you can review this errors handling and follow-up on ingest failure Events to troubleshoot and resolve them. When you have followed-up on an ingest validation failure Event and resolved it, you can select Mark as actioned and remove it from the Ingest Events list:
Note: Submit Measurements API call does NOT throw an Error. If you use the Submit Measurements call to the Ingest API and the call is invalid due to one of the ingest validation failure errors, please be aware that the submission is not immediately rejected:
The API response will be:
"result" : "accepted".This is by design - the API response refers to the pre-enrichment stage of the usage data ingest process. This synchronous stage is kept as simple as possible to avert the possibility of data loss as the first priority. The asynchronous enrichment stage can then carry out checks and reject any invalid usage data measurements.
Note: Ingest Failure Events also show as System Alerts. For some ingest.validation.failure Events - for example the MissingField ingest failure Event - you'll also see a corresponding System Alert created for your Organization on the Alerts page. You can use the Alert as an alternative way to inspect details of the failure to follow-up and troubleshoot the issue. See Viewing and Managing System Alerts for more details.
Reviewing Ingest Events
To review and resolve ingest failure Events:
1. In the Console, select Usage>Ingest Events. The Ingest Events page opens and lists any ingest failure Events generated for data submission measurement errors that have occurred in your Organization, and which have not yet been actioned:
You can read-off the details of the Event:
NAME. The name of the Event.
DATE. The date and time of the Event.
TYPE. The error that occurred and caused the ingest failure:
UndefinedMeter. The Meter Code used does not reference an existing Meter:
Consequence: The submitted usage data measurement is not ingested.
MissingField. A field required to successfully ingest the measurement is missing. For example, this could be a Data Field defined for the Meter:
Consequence: The submitted usage data measurement is not ingested.
DuplicateUid. A non-unique, duplicate UID was used for the measurement submission:
Consequence: The submitted usage data measurement is not ingested.
DuplicateField. The measurement contained data fields with the same code in different categories.
Consequence: The submitted usage data measurement is not. ingested
A Download log button is provided for each failure Event.
2. If you want to follow-up and investigate the ingest failure, you can select the Download log button for the Event:
A error log text file in JSON format is downloaded.
3. Open the downloaded log file. The file contains details of the error that caused the ingest failure, along with the original measurement to help identify and resolve the issue. For example:
In this example, the downloaded log file shows an ingest failure caused by an UndefinedMeter. We can see that the Meter Code used is identified as not belonging to an existing Meter in the current Organization.
Note: Delay in Ingest Failure Events Showing? There can be a delay of one to two minutes before an ingest failure shows on the Ingest Events page.
Tip: Batch Submissions? If you submit a set of usage data measurements as a batch submission, then all good measurements are ingested but any that provoke a, an UndefinedMeter, MissingField or DuplicateUid error are blocked and do not get ingested.
Marking Ingest Events as Actioned
When you have followed-up and resolved an Ingest Event, you can mark it as actioned to remove it from the list.
To mark an Ingest Event as actioned and remove it from the list:
1. Select the checkbox of the Event you want to mark as actioned.
2. Select the Mark as actioned button. A confirmation popup appears:
3. Select Yes to confirm. The page refreshes and the checked Event is removed from the Events list.
Tip: Mark multiple Events as actioned? You can check-off multiple Events to mark them all as actioned at the same time.
4. If you want to check again on Events that you've already actioned, enable the Include actioned events? switch. The page refreshes and any actioned Events are added back into the Events list, and are clearly marked as ACTIONED:
