4.1.  Get Datasets from one Datasource

GET and POST GetDatasets

Description:  Return the metadata describing the datasets in one datasource.

API Usage:   This endpoint uses 1 API call per 100 (or part thereof) datasets returned.
Please refer to section 8.2 for more details on API rate limits.

GET Sample Request:

				GET https://[server]/GetDatasets?SessionToken=YOUR_ACCESS_TOKEN&Datasource=ECBFX&Page=7&Rows=2&ShortRecord=false

POST Sample Request:

				https://[server]/GetDatasets?SessionToken=YOUR_SESSION_TOKEN
				{
					"SessionToken":"YOUR_SESSION_TOKEN",
					"Datasource":"ECBFX",
					"CaseSensitive":true,
					"Rows":2,
					"Page":7,
					"ShortRecord":false,
					"SortColumns":"Symbol,Description",
					"Filter":"Dollar -Canadian",
					"ValuesSince":"2018-01-01",
					"ReturnAccess": true
				}
CategoryDS Datasources
  • A CategoryDS is a datasource whose datasets are grouped into data categories. The subscriber must be authorised to access each individual data category.
  • A CategoryDS is identified when the result object parameterIsCategoryDSis set to true.
  • TheDatacategoryparameter must be used in all data requests to a CategoryDS datasource UNLESS the request parameter ' IgnoreCategories' is set to true.
  • In GET requests Datasource, [Datacategory] and Symbol are combined into one 'Series' string.
    e.g. Symbols[]=Datasource/[Datacategory]/Symbol&Symbols[] = ...
  • In POST requests, the data category is inserted between the datasource and the symbol
    e.g. ” {“Datasource”:“AAA”, “Datacategory”:“BBB”, “Symbol”:“CCC”}, { "Datasource": ….


Request Parameters

Parameter Type Description
1 SessionToken String A session token from the GetSessionToken endpoint.
2 Datasource String The datasource ID.

Optional Request parameters

Parameter Type Default Description
3 Filter String Return only datasets containing specified words or phrases in the symbol and name columns only.
See the Filters section below for more information.
4 CaseSensitive Boolean false If true, the Filter parameter above will be case sensitive.
5 SortOrder String “asc” The result column sort order. Values can be ‘asc‘ for ascending or ‘desc‘ for descending.
6 SortColumns String “Symbol” The column names used to sort the results. Valid column names are: "Symbol", ‘"Name"’ or ‘EndDate’. Use a comma to separate the column names e.g: “EndDate, Name, Symbol”. The DataCategory parameter is ignored for non CategoryDS datasources.
See the notes section above for information about CategoryDS datasources.
7 IgnoreEmpty Boolean false Ignore datasets that contain no prices or values.
8 ShortRecord Boolean false If true the result does not contain the Description parameter and Additional object.
9 CategoryFilter String Applies to CategoryDS datasources only.
The CategoryFilter is a CSV string of category names used to filter the results to include only the datasets from data categories named in the filter e.g. AA, BB, CC.
See the "CastegoryDS datasources" notes section above for more information about CategoryDS datasources.
10 DatasetFilter String Allows you to filter results based on the actual values in the named dataset columns.
You can filter on multiple dataset columns in one request. See the ‘DatasetFilter' section below for more information.
11 Page Integer 1 The requested page to display from from result (see rows below).
12 Rows Integer 100 Number of results per page. Default is 100. Maximum is 5000.
13 ValuesSince String “Earliest” Only return datasets that have values after this date. Use the word “Earliest” or a date in “YYYY-MM-DD” format e.g: “ValuesSince”:”2018-01-01″.
14 DateFormat String “YYYY-MM-DD” The date format to use in the request and result. See the ‘DateFormat Parameter‘ section below for details.
15 ReturnAccess Boolean false If true, a UserAccess object with information on each dataset is returned.
If the result dataset parameter Subscription is set to “None”, the UserAccess object is not returned as no subscription exists.In a CategoryDS datasource, the UserAccess object is located in the UserCategoryList object which is in the DetailsDS object.
In a non CategoryDS datasource, the UserAccess object is located in the Details object.
See the ‘UserAccess Object’ table below for more information.

Filter

Filters are applies to the Symbol, Name and Description columns only. They allow you to restrict the result to those datasets that contain (or exclude) certain words or phrases. Filters use the 'CaseSensitive" parameter setting. Phrases (multi word strings) are always enclosed in double quotes.

To use a double quote to identify the start and end of a phrase, you need to prefix it with the escape character “\” . e.g: \”Crude Oil\” will filter the results to contain the phrase “Crude Oil”.

You must 'escape' the following characters (by preceding them with a "\" backslash character.")

Character Description
, Comma
- Minus
\ Backslash
To embed a double quote (") in a string or phrase, it must be preceded by 3 escape characters "\\\" .

You can exclude datasets containing words and phrases by prefixing the word or phrase with a minus ‘–‘ character e.g: -Crude or – \”Crude Oil\”.

Filter examples:

“Filter”:” DOLLAR YEN \”FOB U\” ” will return datasets containing the words DOLLAR and YEN and phrase “FOB U”
“Filter”:” DOLLAR YEN -\”FOB U\” ” will return datasets containing the word DOLLAR and YEN but NOT containing the phrase “FOB U”

Tips:

  • You can use the CaseSensitive parameter to enhance the Filter function.
  • The CategoryFilter parameter returns datasets from named categories.
  • The DatasetFilter parameter filters based on the contents of dataset columns.
  • You can use any combination of the three filters in a single request.


DatasetFilter

Sometimes you may need to return datasets based on the values that they contain. For example, datasets that have values before or after a certain date, have a minimum number of values or are reported in specific frequency.

The “DatasetFilter” filter allows you filter based on the values in named columns (dataset fields). Each filter is made up from 3 items - a column name (enclosed in square “[]” brackets), a filter operator and a filter condition. See the table below for details.

You can apply multiple dataset column filters in a single request by separating the filters with a comma character.

To create a dataset column filter, you use the column name (as returned in the “Datasets Object” in the “Dataset” and “User Favourite” endpoints). Column names are not case sensitive but must be enclosed in square brackets. For example [Name] is used when selecting the “Name” column.
After selecting the column name, select an operator that matches the column datatype (see the table below) and a filter condition See the “Filter Examples” section below.

Important: You must "escape" any comma, minus, forward slash and double quote characters in any search string or phrase. Please refer to the Filter section above for examples of how to "escape" these characters in strings. The columns used can only be of String, Boolean ,Dates and Integer types. This means that the array and object type columns “Bates”, “Bateindex”, “UserAccess” and “Additional” cannot be used in the DatasetFilter.


Type Operator Description
String $
 		$ means “contains” and is NOT case sensitive.
String filters apply the “Simple Filter” rules (see above) to the values in the named column.

Spaces are optional and string values should not contained in double quotes as these are used to define phrases
e.g. "DatasetFilter":{Name]$“\”Crude Oil\” –Brent.

Boolean = Values can only be true or false with no quotes.
Date and Integer =
<
>
<>
= means equal to.
< means less than.
> means greater than.
<> means not equal to.

Dates and integers do not use quotes.
e.g.
[End Date]>2020-12-21
[Values] >200

Column Filter Examples

Filter 1. Uses one column in the filter .
Description: Filter the results so column “Name” contains the word “Crude” but not the word “Brent”.
"DatasetFilter":"[Name]$CRUDE –BRENT"


Filter 2. Uses two columns in the filter.
Description: As filter 1 above and the contents of the “Values” column are greater than 100.
"DatasetFilter":"[Name]$CRUDE –BRENT, [Values]> 100"


Filter 3. Using two columns with string phrase example.
Description: Filter the results so column “Name” contains the word “CRUDE” but not the phrase “Brent Oil” and the “EndDate” column contents are greater than 2021-12-17. A date columns uses the date format as defined in the JSON request. The default format is “YYYY-MM-DD” Note the double quotes marking the phrase are "escaped".
Remember that strings and phrases are not case sensitive.
"DatasetFilter":"[Name]$CRUDE –\"Brent Oil\", [Values]> 100, [EndDate]> 2021-12-17"

Filter 4. String and phrase search containing double quotes.
Description: Searches for the word "Brent" and phrase "is a "Crude" product." .
"DatasetFilter":"[Name]$CRUDE \"is a \\\"Crude\\\" product.\""




DateFormat Parameter

You can control the format of dates that are used in the request and returned in the result.

If used, the date separator is the character between the day, month and year values and can be one of 4 characters:

“-” – hyphen or dash
“/” – forward slash
“.” – dot or full stop
” ” – space

Format Result Dates
“YYYY-MM-DD” Formatted as 2018-01-22. (default).
“YYYY-DD-MM” Formatted as 2018-22-01.
“YYYYMMDD” Formatted as 20180122.
“YYYYDDMM” Formatted as 20182201.
“MM-DD-YYYY” Formatted as 01-22-2018.
“DD-MM-YYYY” Formatted as 22-01-2018.
“EXCEL” Formatted as numbers where 1/1/1976 = 27760.
“JULIAN” Formatted as the number of days since ’12h Jan 1, 4713 BC’.
Note

In the examples above we use the “-” date separator character. You can replace this character with any of the other three separators characters. If you require 2 character years, replace “YYYY” with “YY” in the formats above.


Response Parameters

Parameter Type Mandatory Description
1 ServerDateTime String Yes The server UTC date and time as an ISO 8601 string.
2 TimeStamp Float Yes The server date and time as a Julian date.
3 ExecutionTime Float Yes The number of milliseconds to process the request.
4 Result [Object] No* If the request is a success, a Result object is returned.
See the ‘Result Object‘ table below for details.
5 Errors [Object] No* If the request fails, an 'Errors' object is returned.
See the ‘Errors Object‘ table below for details.
Note

A successful API response will return a 'Result' object with the requested payload or an 'Errors' object detailing the issue that caused the error.


Result Object

Parameter Type Description
1 Metadata [Object] The datasource summary data as described in the Metadata table below.
2 Datasets [Object] The datasets returned. See the Datasets object below.

Metadata Object

Parameter Type Mandatory Description
1 Datasets Integer Yes The number of datasets in the datasource.
2 Filter String Yes Any filters used to build the result. See the request Filters parameter for details.
3 CategoryFilter String No Applies only to CategoryDS datasources.
A CSV string of category names used to filter the results to include only the datasets from these data categories.
This parameter is case sensitive.
4 SortOrder String Yes The result sort order based on SortColumns property. Values can be ‘asc’ or ‘desc’ (see request).
5
SortColumns
String Yes The column names used to sort results by. Valid column names are: Symbol, CategoryName, Name and Description. Use a comma to separate column names e.g: “Symbol,Description”.
6 CaseSensitive Boolean Yes If true then case-sensitive is applied to the filter (see request).
7 Page Integer Yes The page number returned in the result (see request).
8 PageCount Integer Yes Total number of possible pages in result based on available datasets and the rows value below.
9 Rows Integer Yes The number of rows per page in the result (see request).
10 ShortRecord Boolean Yes If true, the result does not contain the Description parameter and Additional object.
11 ValuesSince Date No Returns only datasets that have values since this requested date (see request).

Datasets Object

Parameter Type Mandatory Description
1 Datasource String Yes The datasource ID.
2 DataCategory String No If true, this is a CategoryDS datasource (result ISCategoryDS is set to true) and the DataCategory result will contain the data category name.
3 Symbol String Yes The dataset ID (market identifier, symbol, ID or code).
4 Name String Yes Text string describing the dataset
5 Description String No If available, additional data describing the dataset.
6 Frequency String Yes The frequency (day, week etc.) in which the data is reported.
7 Currency String Yes The currency a unit the dataset is reported in as an ISO three character currency code.
8 Unit String Yes The unit a dataset it reported ( MT, BBL, GALL etc).
9 Decimals Integer Yes The recommended number of decimal places to use when displaying the dataset values.
10 Bates Array Yes An array of price data column ‘bate’ names e.g: “Bid”,”Ask”, “High”,”Low”.
11 Bateindex Array Yes An array of price data column numbers that correspond to the bates array above e.g: “BateIndex”: [ 0, 1, 2 ]
12 StartDate String Yes The first available date in the dataset with a value.
13 EndDate String Yes The most recent date available in the dataset with a value.
14 Values Integer Yes Total number of value/price records in the dataset.
15 Corrections Integer Yes The number of value/price corrections available for the dataset.
16 Premium Boolean Yes If true, this is a premium subscription datasource that is not included gratis as part of a service.
17 Favorite Boolean Yes Is this dataset in the users favorite datasets list?
18 IsCategoryDS String Yes If true the datasource is a CategoryDS datasource. A CategoryDS is a datasource whose datasets are grouped in data categories. The user must be authorised to access individual data categories. See “CategoryDS datasources” note above.
19 Subscription String Yes The Subscription parameter is only returned when the request parameter ReturnAccess is set to true.
It returns the type of datasource access available with the current subscription.
There are 4 options: “None”, “Active”, “Inactive” and “Simulated”.
Please note that the option “Simulated” means that the values are fictional and are generated for demonstration purposes only.
20 UserAccess [Object] No If Subscription equals “None” the UserAccess object will not be returned in the result.
If requested, the UserAccess object returns details of the access allowed with the current subscription.
See the “User Access Object” table below for details.
21 Additional [Object] No The Additional object allows more dataset specific information to be provided as additional JSON parameters.
Please refer to any individual datasource documentation and the JSON returned from the Additional object for more information.
An example of conversion factors in the Additional object is:

“Additional”: {
“Conversions”: [
{
“ConvertTo”: “BBL”,
“ConvertOperator”: “/”,
“ConvertValue”: “7.45”
}
]
}

UserAccess Object

If the user has (or had) a subscription to the datasource (or data category), the UserAccess object reports on the range allowed for the dataset with the current subscription.
The Starts and Endsparameters show the date range that the user can access.

Parameter Type Description
Starts String Date of the earliest available data for the current subscription.
Ends String Date of the latest available data for the current subscription.
Notes

  • There may be a greater range available in the datasource than the user has permission to access.
  • The UserAccess object is present only if request parameter ReturnAccess is set to true and Subscription is not equal to “None”.
  • The request parameter DateFormat controls the format of the result in the Starts and Ends parameters.

Errors Object

Parameter Type Description
1 Status Integer HTTP status code.
2 Detail String Description of the status code error.

Possible Status codes in the 'Errors' object

Code Description
400 Datasource field is empty or too many rows requested (500 maximum).
401 Invalid access token.
402 Unauthorized.
403 Premium subscription required.
404 Unknown datasource or symbol.
410 Session Token expired.
413 Payload too large.
Note

This is not an exhaustive list and additional error codes and descriptions may be returned. The Errors object Detail field will return request specific error information.

Possible HTTP response codes

This endpoint returns 200 for success.
You can get HTTP client network errors or server errors from any API request. You should check with your network or IT colleagues in case the issue is a local one.
All server generated errors will return an 'Errors' object that will give specific details about the error.
A full list of HTTP codes and their meanings is available here,

Success and fail examples

The sample requests and token using a dummy token for display purposes.

Sample Request:

GET https://[server]/GetDatasets?SessionToken=YOUR_SESSION_TOKEN&Datasource=ECBFX&Page=1&Rows=2&ShortRecord=false

Response body on success:


Response body on fail:

Last updated on October 12, 2024

Datasets Get Datasets from multiple Datasources