Smart Correct

Understand Smart Correct

One of our platform’s superpowers is its ability to flag errored and missing data. Validation is done when data is brought into the platform, whether manually or through inbound integrations. The Data Standards Cloud reviews the data in each row of your dataset against your defined data standards and alerts you to nonstandard fields that needed your attention. 

This enables you to correct data before it’s used in analytics or data storage, as well as to send fixes back to the platform of origin. And with Smart Correct, we make corrections to this errored data as simple as a click.

AI-Enabled Smart Correct

The Smart Correct feature is available to all Claravine customers. Customers who have an AI-enhanced Claravine contract also enjoy AI-enabled Smart Correct, which offers an elevated level of recommendations. Contact your Claravine account manager to learn more.

Smart Correct Data-Entry Issues, Not Template Issues

What looks like erroneous data can be caused by data entry errors or by problems with the underlying template. Smart Correct catches unintentional data entry errors, not errors within the template. This is an important distinction. If there's a template issue, correcting wrong entries as data errors without fixing the template itself would only exacerbate the problem.

For instance, let's say your Claravine template has a "Channel" drop-down list with the values 'Facebook', 'Instagram', and 'Snapchat', but at some point, your Digital Asset Manager's list of "Channel" values changes to 'FB', 'IG', and 'SNAP'. If you forget to update the list field definition in Claravine and upload these new updated values, they will yield errors in Claravine. In this case, it's desirable for you to see these errors and update the list field definition to match your DAM. We don't want Smart Correct to treat the entries as data errors, "correcting" them to values that are, in fact, outdated, and then mark them as fixed.

Smart Correct is designed to approach corrections conservatively, assuming that large corrections are more likely to signal template problems. It will not modify original entries if it deems the modifications too large and too risky. Smart Correct is designed to strike a balance between caution and usefulness.

Smart Correct and Empty Cells

Smart Correct is most effective when there is existing data to work with. When a cell is completely empty, there’s no context for the system to generate a meaningful suggestion.

In most cases, empty cells can’t be corrected. The exception is when the field is part of a drill-down or dependent list, where Smart Correct may be able to infer a value based on related data in the row.

If you're working with rows that contain missing values, be aware that Smart Correct may not provide suggestions for those fields. For the best results, focus on correcting data that’s present but inaccurate or inconsistent, rather than fields that are entirely blank.

Use Smart Correct

When data is brought into pending through an inbound integration or manual upload, the platform highlights (in red) all errored fields that don't adhere to the data standards for that template.

1. Use the filters to select the rows to include in the edit.
    

2. To use Smart Correct, select the Smart Correct # Rows button instead of the green Edit # Rows button.

   

    

3. The Data Standards Cloud will automatically suggest values based on existing rules and taxonomy across errored fields. 

   Whether AI is engaged depends on your organization’s policy and account settings.

   Fields updated by Smart Correct will change to green.
   
    

4. Hover over green fields to see additional detail about the suggested correction.

   
   
    

5. Finalize edits and submit data for processing.

Correction Types

Text Fields

Text fields can contain multiple errors. For example, a field may violate a “Begins with” rule and an exclusion rule. Smart Correct attempts to fix both errors.

Prefixes and Suffixes

This correction is triggered by creating a text field with the “Begins With” rule or “Ends With” rule, and then failing to start or end the field value with the specified characters.

Examples:

Disallowed Numbers

This correction is triggered by creating a text field with the “Exclude - Numbers” rule, and then including numbers in the field value

Note that only two numbers are corrected. If there are more than two numbers, the field is not corrected on the premise that more than two disallowed numbers may indicate that the numbers were intentional and the template needs modification.

Examples:

Disallowed Spaces

This correction is triggered by creating a text field with the “Exclude - Spaces” rule, and then including spaces in the field value

Example:

List Fields: Dropdown Lists

Misspelled or Malformed value

This correction is triggered by small misspellings or bad formatting in a list field cell, resulting in an invalid field value.

Example 1:

Example 2:

Swapped List Field Values

This correction is triggered when a template contains at least two list fields. For a given submission row, two list field cells contain invalid values, and each invalid value exactly matches a single valid value from the other list field.  

Examples:

Abbreviated Values

Note: This is a Claravine AI-enabled capability. It requires an AI Addendum.

This correction is triggered when a cell contains text that is a commonly accepted abbreviation of one of the valid list values, or vice versa.

Note that abbreviated values are only corrected if the resulting change is ±4 characters in length from the invalid value.

Examples:

List Fields: Drilldown and Dependent Lists

Drilldown lists and dependent lists contain multiple columns. A dependent list is a drilldown list in which the very first column is unique. A dependent list is a valid drilldown list and therefore functions the exact same way. Because they have the same behavior, the rest of this section discusses only drilldown lists (it is the more general case), but know that everything here applies to both types of lists.

To correct values in a drilldown, we must first determine what the possible options are in a given column based on the values in the other columns. Consider this drilldown:

And this submission row:

Because Canada was selected for the Country field, we know based on the drilldown definition that we can narrow down Metro to two options: Toronto, or Vancouver.

Next, we need to consider the errored value itself. The user provided Trnto, which is not one of the valid options. The system uses the same correction logic that it uses for dropdown lists to determine that, given Toronto and Vancouver as options, Trnto should be corrected to Toronto.

Misspelled and Abbreviated Fields

Example 1:

Note: Example 1 illustrates a Claravine AI-enabled capability. It requires an AI Addendum.

Example 2:

North Amer doesn’t match any of the valid values for Continent, but the system recognizes that it’s a misspelling of North America and corrects it.

Blank Fields

Example 1:

Because South America and Brazil are selected, the only possible option for Metro is Sao Paolo. The system assumes that the user did not intend to leave this field blank, and corrects to Sao Paolo.

Example 2:

The system can infer that because Continent is South America and Metro is Buenos Aires, there is only one possible Country value. It corrects to that value, which is Brazil.

Multiple Errors

Example:

The system sees that Country must be either Canada or United States. Canada_1 is a misspelling of Canada, so this is corrected. Once Country is known, Metro has only one possible option, Montreal.

Invalid Combinations

Although Brazil is a valid value for Country in the drilldown list definition, it is not valid given that Continent is North America. However, changing Brazil to Canada would then make Buenos Aires an invalid option for Metro. Instead, the system recognizes that the values for Country and Metro are tied to a single possible combination. The system changes Continent to South America because that requires only one field correction, and is more likely what the user intended.

Date Fields

Malformed Dates

This correction is triggered when date field values are provided in a different format than what the template specifies.

Note that this only works when the provided date is still in a reasonably interpretable format. To make a correction, Smart Correct must be able to parse what date the value represents in the first place.

Examples: