Search Knowledge Base by Keyword

Generic Rest XML API

< Back

Overview

The Generic Rest XML API connector is a build-your-own integration for any HTTP API that answers in XML rather than JSON. It is not tied to a vendor. You supply the base domain, the authentication style, the endpoint path for each job and the node in the returned document that holds the records, and ReadyWorks does the calling, paging and parsing.

It is the XML sibling of the Generic Rest JSON API connector. The difference is the parser, not the transport: both drive ordinary REST calls, but this connector runs the XML driver (xml_rest_api.php) and exposes two settings the JSON connector has no use for. Ignore XML Attributes decides whether element attributes are flattened into the staging row or dropped, and API Return Data Node is read as a path into the XML tree instead of a JSON node. Use this connector when the endpoint returns an XML document, and the JSON connector when it returns JSON. The authentication methods offered here are a subset of the JSON connector’s, so an XML API that needs a token-fetch or session-cookie login is not covered by the four methods shipped.

Each job writes into its own staging table in the ReadyWorks estate, named by the job’s Destination Table. From there the data is mapped, reconciled against the rest of the fabric and used like any other source. All three enumeration levels are available, so a job can walk a list of keys returned by one call and issue a child call for each key, up to three levels deep.

The connector is inbound only. It reads from the API and writes to ReadyWorks staging. There are no outbound job fields and no outbound templates, so it cannot be used to push data back to the API.

NOTE: The shipped inbound template is an empty skeleton. It has no endpoint and is disabled. Treat it as a starting point to copy rather than a job that will run.

Connector Properties

Property Value
Identifier XML Rest API
Name Generic Rest XML API
Description Connector for pulling and processing XML data from a REST API link
Job Types Inbound Only
Order 70
Enabled Yes
Locked Yes
Block Update No
Single Authentication No
Windows Only No
Connector Version 2025-03-12
Hooks None
Additional Job Fields None
Image

Authentication Methods

Four authentication methods are shipped, all of them stateless: the credential (if any) is attached to the request itself and there is no separate login call to fetch a session or token.

Method Identifier Base Method Script Order Enabled Config Fields
No Authentication XML Rest API_none XML Rest API_none xml_rest_api.php 10 Yes 4
API-Key XML Rest API_key XML Rest API_key xml_rest_api.php 20 Yes 5
Bearer Token XML Rest API_token XML Rest API_token xml_rest_api.php 30 Yes 5
Basic Authorization: Username / Password XML Rest API_user XML Rest API_user xml_rest_api.php 40 Yes 6

NOTE: Every method sets SSL_VERIFYHOST and SSL_VERIFYPEER to false, so the API’s TLS certificate is not validated. If you are calling an endpoint over the public internet, raise this with your ReadyWorks contact before putting the connection into production.

Method 1: No Authentication (XML Rest API_none)

Connect to a GET method API with no authentication

Sends no credential at all. Use it for endpoints that are open to the network the ReadyWorks collector sits on, for example an internal reporting service behind the firewall, or for a first connectivity test against a public feed. The connection form asks only for the API domain and the temporary working path.

Connection Configuration Fields (4)

Order Label Type Required Default Max Len Tooltip
10 API Domain text Yes 255 Enter API domain of the Connection
30 Server Temporary Working Path text Yes storage/etl5 255 Enter path where temporary files are stored during data processing (exclude starting and ending backslash)
240 Single Authentication radio Yes 0 Choose if authentication is only to be done once and not on each enumeration or pagination loop
250 Additional Job Fields fields_list No Enter list of additional fields to add to staging data for all jobs

Authentication Configuration

cURL options:

Option Authentication Call Data Calls
USERAGENT Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0 Same
SSL_VERIFYHOST false Same
SSL_VERIFYPEER false Same
RETURNTRANSFER true Same
FOLLOWLOCATION true Same
FAILONERROR true Same

Method 2: API-Key (XML Rest API_key)

Connect to a GET method API with an API-Key

Collects an API key into the encrypted auth_value_1 field for endpoints that authenticate with a static key. Confirm on a first run, with Log Raw API Calls turned on, that the key reaches the endpoint in the header or parameter your API expects, because this method’s recipe defines no header of its own.

Connection Configuration Fields (5)

Order Label Type Required Default Max Len Tooltip
10 API Domain text Yes 255 Enter API domain of the Connection
30 Server Temporary Working Path text Yes storage/etl5 255 Enter path where temporary files are stored during data processing (exclude starting and ending backslash)
60 API Key password No 64000 Enter API key of the Connection
240 Single Authentication radio Yes 0 Choose if authentication is only to be done once and not on each enumeration or pagination loop
250 Additional Job Fields fields_list No Enter list of additional fields to add to staging data for all jobs

Authentication Configuration

cURL options:

Option Authentication Call Data Calls
USERAGENT Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0 Same
SSL_VERIFYHOST false Same
SSL_VERIFYPEER false Same
RETURNTRANSFER true Same
FOLLOWLOCATION true Same
FAILONERROR true Same

Method 3: Bearer Token (XML Rest API_token)

Connect to a GET method API with a bearer token

Sends Authorization: Bearer with the value you store in the Bearer Token field. Use this for APIs issuing long-lived personal access tokens or service tokens that you can paste in and rotate manually. It does not fetch or refresh a token, so an endpoint that issues short-lived tokens from a login call is not served by this method.

Connection Configuration Fields (5)

Order Label Type Required Default Max Len Tooltip
10 API Domain text Yes 255 Enter API domain of the Connection
30 Server Temporary Working Path text Yes storage/etl5 255 Enter path where temporary files are stored during data processing (exclude starting and ending backslash)
60 Bearer Token password No 64000 Enter bearer token of the Connection
240 Single Authentication radio Yes 0 Choose if authentication is only to be done once and not on each enumeration or pagination loop
250 Additional Job Fields fields_list No Enter list of additional fields to add to staging data for all jobs

Authentication Configuration

Headers sent on the authentication call:

Header Value Base64
Content-Type: application/json Off
Authorization: Bearer %auth_value_1% Off

cURL options:

Option Authentication Call Data Calls
USERAGENT Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0 Same
SSL_VERIFYHOST false Same
SSL_VERIFYPEER false Same
RETURNTRANSFER true Same
FOLLOWLOCATION true Same
FAILONERROR true Same

NOTE: The Bearer Token field is not marked required, so a connection can be saved with the field empty. The problem then appears as a 401 on the first scheduled run rather than as a validation error at save time.

Method 4: Basic Authorization: Username / Password (XML Rest API_user)

Connect to a GET method API with a user/password

Sends Authorization: Basic with the username and password Base64-encoded together. This is the right choice for the many internal appliances and legacy XML services that still accept HTTP basic authentication.

Connection Configuration Fields (6)

Order Label Type Required Default Max Len Tooltip
10 API Domain text Yes 255 Enter API domain of the Connection
30 Server Temporary Working Path text Yes storage/etl5 255 Enter path where temporary files are stored during data processing (exclude starting and ending backslash)
40 Username text No 1024 Enter username of the Connection
50 Password password No 64000 Enter password of the Connection
240 Single Authentication radio Yes 0 Choose if authentication is only to be done once and not on each enumeration or pagination loop
250 Additional Job Fields fields_list No Enter list of additional fields to add to staging data for all jobs

Authentication Configuration

Headers sent on the authentication call:

Header Value Base64
Content-Type: application/json Off
Authorization: Basic %user_name%:%user_pass% On

cURL options:

Option Authentication Call Data Calls
USERAGENT Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0 Same
SSL_VERIFYHOST false Same
SSL_VERIFYPEER false Same
RETURNTRANSFER true Same
FOLLOWLOCATION true Same
FAILONERROR true Same

Inbound Job Fields Enabled (45)

The inbound job form exposes the full generic REST field set plus the two XML-specific controls, Ignore XML Attributes and the XML-aware API Return Data Node.

Order Label Type Required Default Tooltip
10 Job Description text Yes Enter description of the Job
20 Job Schedule lookup Yes Daily Select frequency Job should run
30 Enabled radio Yes Yes Choose if Job is enabled
40 Use Unparsed Data radio No No Choose if you wish to parse the incoming data, or leave it in its raw format
50 Log Raw API Calls radio Yes No Choose if raw API calls are logged
70 Destination Table text Yes Enter name of the destination table
80 Data Identity text No Enter identity of the Job
120 Append New Data to Existing Tables radio No No Choose if new data will append to the existing destination table, or will create a new destination table
130 Fields to Index text No Enter fields to index
170 Method Type select Yes GET Select method type of the Job
190 API End Point text No Enter API end point of the Job
200 API Data Parameters fields_list No Enter API data parameters of the Job
210 Body Data Sending Method select Yes JSON Encoded Data Select method for sending the body data
220 Raw JSON Body Data textarea No Enter the raw JSON body to send. Do not use body selections in the API Parameters if this is used.
230 API Return Data Node text No Enter API return data node of the Job
240 Ignore XML Attributes radio No No When parsing XML ignore field attributes and do not include them in the data
250 Pagination Type select No None Select request paging type of the Job
270 API Pagination Node text No Enter API pagination node of the Job
280 Pagination Offset Start text No Enter request paging offset start of the Job
290 Request Limit text No Enter request limit of the Job
300 Pagination Termination Node text No Enter request paging termination node and value of the Job
310 Maximum API Calls text No Enter maximum number of API calls for the Job to make
320 Enumeration 1 Type select No None Enter enumeration level 1 type
330 Enumeration 1 ETL Job lookup No Select ETL job for the enumeration 1 values
340 Enumeration 1 Data Type data_type No Select the data type and fields for enumeration 1 values
350 Enumeration 1 Keys End Point text No Enter enumeration 1 keys end point of the Job
360 Enumeration 1 Node text No Enter enumeration 1 node of the Job
370 Enumeration 1 Fields fields_list No Enter enumeration 1 fields of the Job
380 Enumeration 2 Type select No None Enter enumeration level 2 type
390 Enumeration 2 ETL Job lookup No Select ETL job for the enumeration 2 values
400 Enumeration 2 Data Type data_type No Select the data type and fields for enumeration 2 values
410 Enumeration 2 Keys End Point text No Enter enumeration 2 keys end point of the Job
420 Enumeration 2 Node text No Enter enumeration 2 node of the Job
430 Enumeration 2 Fields fields_list No Enter enumeration 2 fields of the Job
440 Enumeration 3 Type select No None Enter enumeration level 3 type
450 Enumeration 3 ETL Job lookup No Select ETL job for the enumeration 3 values
460 Enumeration 3 Data Type data_type No Select the data type and fields for enumeration 3 values
470 Enumeration 3 Keys End Point text No Enter enumeration 3 keys end point of the Job
480 Enumeration 3 Node text No Enter enumeration 3 node of the Job
490 Enumeration 3 Fields fields_list No Enter enumeration 3 fields of the Job
500 Pause Enumeration text No Enter values for pausing enumeration every X calls for Y seconds
510 Kill Enumeration on Error radio No No Choose to stop the enumeration loop on an error or continue processing the loop
520 Additional Fields fields_list No Enter list of additional fields to add to staging data
550 Retry Pause text No Enter values for retrying a failed API connection. Retry X times, waiting Y seconds between each try, with an API timeout of Z seconds.
560 Order text Yes Enter order of the Job

NOTE: Pagination is off by default (Pagination Type = None). Set it before pointing a job at a collection endpoint, or the job takes whatever the first response contains and stops.

NOTE: Ignore XML Attributes is set to No by default, so attributes on an element are parsed into the staging row alongside its child elements. Set it to Yes when the attributes are schema noise such as namespace or type markers and you do not want a column for each.

Inbound Job Templates (1)

One inbound template ships, and it is a blank starting point rather than a working job.

# Job Description Destination Table API End Point Enabled What It Pulls
1 Rest GET API api_data Not set No An empty GET skeleton with no endpoint set, writing to a staging table called api_data. Copy it, fill in the API End Point and Destination Table, set the API Return Data Node to the element that repeats once per record, and enable it.

Job Template Configuration

Settings

Setting Value
Order 10
Enabled No
Job Schedule Daily (15 1 * * *)
Destination Table api_data
Data Identity api_data
Allow Empty Table Yes
Append Files to Same Destination Table No
Append New Data to Existing Tables No
Use Unparsed Data No
Convert UUID-Keyed Objects to Rows No
Ignore XML Attributes No
Log Raw API Calls No
Method Type GET
Body Data Sending Method JSON Encoded Data

NOTE: The template ships disabled (Enabled = No). Nothing runs until you configure a job and turn it on.

Outbound Job Templates (0)

This connector is inbound only, so no outbound templates are shipped and the outbound job form is not available.

Tags: