File Format

delimiters all files contain the following delimiters column pipe ( | ) row carriage return and line feed header row the first row in all files is a header row this contains the name of each column, separated by a column delimiter metadata columns change bit the first column in all incremental feed files is the change bit this denotes whether the row record is a newly created record change bit c an updated record change bit u a deleted record change bit d when a row record has the change bit d , all other columns, except the primary key id, for the record will be null the change bit is only present in incremental feed files it does not appear in full feed files deleted records do not appear in full files if you intend to load from both our full and incremental files, it is recommended to process any outstanding incremental files before processing the full files this will ensure any deletions are applied first record updated timestamp the second column in all incremental feed files is the record updated timestamp this indicates the datetime when the row record was last updated the format is yyyy mm ddthh\ mm\ ss mss+0000, where mss is a three digit millisecond and +0000 represents the utc time zone below is an example 2023 02 08t16 52 52 062+0000 the record updated timestamp is only present in incremental feed files it does not appear in full feed files data types in addition to the above metadata fields, the data fields within a feed file can be one of the following types boolean all boolean values are represented as a string literal true or false dates all dates in the feed files are represented as iso date strings, in the format yyyy mm dd as shown below 2023 01 01 dates can be partia l, with values to the right of yyyy omitted year only yyyy – (e g 2000 ) year and month yyyy mm (e g 2000 01 ) strings we recommend storing string fields with a varchar(max) or text type in sql server 2022 and varchar with no length specified, or text type in postgresql