Enable and configure a Secure File Transfer Protocol (SFTP) integration for your Case IQ application on the Data Import page under the System tab in Settings. With an SFTP integration, administrators can import data from another application (such as the organization’s human resources information system) to populate profile records in Case IQ. This article covers the steps to enable an SFTP integration by adding a data import record. If you have already set up a data import and want to schedule the process, see the Schedule a Data Import section of this article. 

Data File Format

Data files uploaded to Case IQ's SFTP server must meet the following requirements:

• All data for an import must be included in one data file. 
  • For a recurring SFTP data import, you can upload a new data file for each scheduled import.
  • You can import a maximum of 25,000 records per import.
• Each profile's data must be in one row in the data file, meaning that there is one profile per row. Likewise, one profile field per column.
• Data files must be in .csv format. We use RFC 4180 standards for .csv files, which you can review in the Common Format and MIME Type for Comma-Separated Values (CSV) Files memo.
• Data file names must begin with the "Unique file name prefix" set in the SFTP data import rule action (see the Schedule a Data Import section).
• The maximum file size is 15MB.

Add a Data Import

To add a data import, click the Add Data Import button on the Data Import page. The New Data Import form includes multiple sections to fill in. First, complete the following fields in the Data Import Parameters section:

• Name: enter a name to refer to the data import.
• Import Entity: choose "Profile". 
• Update Method: choose how the system will import the data into your application. The system will use the unique identifier that you set in the Field Mapping section, such as an employee ID or email address, to match rows in the data file with profiles in Case IQ. A data file row and Case IQ profile with the same value in the unique identifier field will be considered the same profile. Select one of the following options:
  • Replace all existing records: the system will only update profiles with a matching unique identifier in the data file.
  • Update existing and insert new records: the system will update profiles with a matching unique identifier in the data file. Then, it will add a profile for any rows in the data file that do not have a matching unique identifier in Case IQ.
  • Only insert new records: the system will add a profile for any rows in the data file that do not have a matching unique identifier in Case IQ. It will not update any existing profiles in Case IQ.
• Error Notification Recipients: enter an email to receive error notifications if the system cannot process an import.

Next, you can tell the system what data to populate in profile fields by filling in the Field Mapping section.

1. Click the "+ Field Mapping" button. 
2. Search for and select a profile field in the "Target field" dropdown. You can select any dynamic or static Case IQ profile field as a target field. You can only map one data file column to a field.
3. When you select a target field, the “Data file column header” field will be automatically populated with the target field’s “Caption” as a placeholder. You can edit the “Data file column header” so it matches the column header in your data files.
4. Select whether the field is required for a profile. If a row in the data file does not have a value for a field you set as required, the system will not add a profile or update an existing profile for the row.
5. Click the Options () button for the row. You can set one row as the unique identifier by selecting the "Set as unique identifier" option.
   1. Every profile must have a different value for their unique identifier, such as an email address or employee ID.
   2. You can only select one unique identifier for a data import.
   3. The unique identifier acts as each profile's ID. It tells Case IQ how to identify a single profile record. During an import, the system will look for data file rows and Case IQ profiles with the same unique identifier. Rows and profiles with the same unique identifier are considered the same profile.
6. Continue to map more fields in your data file by clicking the "+ Field Mapping" button. You can map up to 40 fields per data import.
7. When you are done mapping fields, click the Save button to save your data import. 

After filling in the Data Import Parameters and Field Mapping sections, click the Save button.

Edit a Data Import

You can update a data import at any time from the data import's page:

1. Click the Edit button. 
2. The data import's page will now be in edit mode. You can now update the data import's parameters and field mapping.
3. When you have completed your changes, click the Save button.

Delete a Data Import

If you need to permanently remove a data import, you can delete it by clicking the Delete button on the data import's page. If you added a "Schedule Process" rule and "SFTP Data Import" action for this data import, the system will not be able to run the data import when scheduled. You can edit the "Schedule Process" rule to change the data import. 

Schedule a Data Import

After configuring your data import, you can schedule when the system will run it by adding a "rule". In Case IQ, rules are configured by system administrators to automatically execute tasks. You add a "Schedule Process" rule to set when the system will run your data import. Use the following steps to add a "Schedule Process" rule and "SFTP Data Import" action. You can see details on other types of rules in Add a Rule.

1. In Settings, navigate to the Rules page under the Workflow tab. 
2. Click the Add Rule button.
3. The New Rule form will be loaded. In the "Name" field, enter a descriptive name for your rule.
4. In the "Rule Type" field, select "Schedule Process".
5. The "Frequency" and "Trigger Time" fields will now be displayed for you to fill in. 
   1. Frequency: choose the frequency with which the system should run the data import. You can choose from the following options: daily, daily (work week), weekly, and monthly. If you select "Daily (work week)", the process will be triggered every Monday, Tuesday, Wednesday, Thursday, and Friday.
      1. "Days of the Month" or "Days of the Week": if you set the "Frequency" to "Weekly" or "Monthly", a "Days of the Week" or "Days of the Month" field will be displayed, so you can choose the days within the week or month to run the process. You can choose multiple days.
   2. Trigger Time: set the time that the system should run the data import. The system may not use your timezone. You can check the system timezone on the About page under the System tab in Settings.
6. Click the "+" button in the Actions section and select "Add SFTP Data Import".
7. The following fields will be displayed in the New SFTP Data Import form for you to fill in:
   1. Data Import: select the data import you want to schedule. You can only select data imports that have already been added into the system.
   2. Unique Filename Prefix: enter a unique set of characters you will add to the beginning of data file names for this import. Case IQ uses this unique file name prefix to identify the files to import into your application when this rule is triggered. For example, if you are importing information on your organization's personnel into Case IQ profiles, you could use "staff", "employees", or "contractors" as the unique file name prefix. Do not include spaces or special characters.
8. When you fill in the unique file name prefix field, Case IQ will populate what your data file names should look like following our file naming conventions in the "Data file name preview" field, i.e. "[unique file name prefix]_{arbitraryId}.csv". 
   1. Make sure the data files exported by your system are named following this format. All data file names must begin with the unique file name prefix for the import followed by an underscore. For example, if the unique file name prefix is "employees", your data file names must begin with "employees_".
   2. Any text following "[unique file name prefix]_" in a data file name is called an "arbitrary ID". Case IQ will not use the "arbitrary ID" to identify data files, so you can include any text as the arbitrary ID.
   3. Each data file for the data import can have a different arbitrary ID. Arbitrary IDs can be helpful for users to identify different data files in a recurring data import. For example, the arbitrary ID can be a timestamp when the data file was exported by your system. 
9. After noting the data file name preview, click the Save button in the New SFTP Data Import form. 
10. Click the Save button for the New Rule form.