API: ETL

Endpoints for getting, creating, modifying and deleting ETL Connectors, Connections and Jobs. Endpoints include:

	etl/connectors	Returns a list of all available ETL connectors and their authentication methods.
	etl/connectors/{needle}	Search connectors using the needle.
	etl/activateConnection/ {authentication_id}	Returns information about a specific connector.
	etl/activateConnection/ {authentication_id}	Activates a new connection for a specific connector.
	etl/connections	Returns a list of all active ETL connections.
	etl/connections/{needle}	Search connections using the needle.
	etl/connection/{connection_id}	Returns information on a specific connection.
	etl/connection/{connection_id}	Updates a connections data.
	etl/connection/{connection_id}	Deletes a specific connection along with all of it’s jobs, logs, and staging tables.
	etl/inboundFields/{job_id}	Get a list of all external dynamic fields used for an inbound ETL job that can be set before the job is ran.
	etl/inboundFields/{job_id}	Sets the external dynamic field values to be used for an ETL inbound job.
	etl/job/{job_id}	Returns the status of the last run of a job.
	etl/job/{job_id}	Runs an ETL job in the background.

etl/connectors Returns a list of all available ETL connectors and their authentication methods.

URL:https://webroot/api/v1/etl/connectors

Parameters

Name	Description
Authorization string (header) * required	Default value: Bearer {token}

Responses

Code	Description
200	Example Value
	application/json
	{ "data": [ { "connector_name": "Microsoft Intune", "authentication_name": "MS Graph Authentication", "authentication_id": "INTUNE_ms_graph", "authentication_description": "Connect to Intune using MS Graph API", "connector_description": "Connector for processing Microsoft Intune data using MS Graph.", "connector_version": "2024-08-31", "connector_id": "INTUNE" }, { "connector_name": "Service Now (SNOW)", "authentication_name": "Username and Password", "authentication_id": "SNOW_user", "authentication_description": "Connect to a Service Now connector with a username and password", "connector_description": "Connector for pulling Device, User, and Service Request data from ServiceNow", "connector_version": "2024-08-31", "connector_id": "SNOW" }, { "connector_name": "Azure Active Directory", "authentication_name": "MS Graph Authentication", "authentication_id": "AZURE_AD_ms_graph", "authentication_description": "Connect to an Azure Active Directory using MS Graph API", "connector_description": "Connector for processing Azure Active Directory data using MS Graph.", "connector_version": "2024-08-31", "connector_id": "AZURE_AD" } ] }

Headers

Name	Type
Date	string
Server	string
X-Powered-By	string
Content-Length	string
Keep-Alive	string
Connection	string

etl/connectors/{needle} Search connectors using the needle. Returns an empty set if the needle is not found.

URL:https://webroot/api/v1/etl/connectors/{needle}

needle = String

Example:https://webroot/api/v1/etl/connectors/graph

Parameters

Name	Description
Authorization string (header) * required	Default value: Bearer {token}

Responses

Code	Description
200	Example Value
	application/json
	[ { "connector_name": "Generic Rest JSON API", "authentication_name": "MS Graph Credentials GET Bearer Token", "authentication_id": "JSON Rest API_ms_graph", "authentication_description": "Connect to a MS Graph method API with bearer token.", "connector_description": "Connector for pulling and processing JSON data from a REST API link", "connector_version": "2024-08-31", "connector_id": "JSON Rest API" }, { "connector_name": "Microsoft Intune", "authentication_name": "MS Graph Authentication", "authentication_id": "INTUNE_ms_graph", "authentication_description": "Connect to Intune using MS Graph API", "connector_description": "Connector for processing Microsoft Intune data using MS Graph.", "connector_version": "2024-08-31", "connector_id": "INTUNE" }, { "connector_name": "Microsoft Teams", "authentication_name": "MS Graph Authentication", "authentication_id": "TEAMS_d363", "authentication_description": "Connect to an Teams using MS Graph API", "connector_description": "Connector for processing Microsoft Teams data using MS Graph.", "connector_version": "2024-05-26", "connector_id": "TEAMS" }, { "connector_name": "Azure Active Directory", "authentication_name": "MS Graph Authentication", "authentication_id": "AZURE_AD_ms_graph", "authentication_description": "Connect to an Azure Active Directory using MS Graph API", "connector_description": "Connector for processing Azure Active Directory data using MS Graph.", "connector_version": "2024-08-31", "connector_id": "AZURE_AD" } ]

Headers

Name	Type
Date	string
Server	string
X-Powered-By	string
Content-Length	string
Keep-Alive	string
Connection	string

etl/activateConnection/{authentication_id} Returns information about a specific connector, including its authentication methods, their required fields and default values, and its inbound jobs. The endpoint requires an authentication_id variable.

URL:https://webroot/api/v1/etl/activateConnection/{authentication_id}

authentication_id = String

Example:https://webroot/api/v1/etl/activateConnection/AZURE_AD_ms_graph

You can find the authentication_id by using the etl/connectors endpoint:

{
    "connector_name": "Azure Active Directory",
    "authentication_name": "MS Graph Authentication",
    "authentication_id": "AZURE_AD_ms_graph",
    "authentication_description": "Connect to an Azure Active Directory using MS Graph API",
    "connector_description": "Connector for processing Azure Active Directory data using MS Graph.",
    "connector_version": "2024-08-31",
    "connector_id": "AZURE_AD"
}

Parameters

Name	Description
Authorization string (header)	Default value: Bearer {token}

Responses

Code	Description
200	Example Value
	application/json
	{ "required_fields": { "label": "", "source_server": "https://graph.microsoft.com", "data_object": "storage/etl5", "auth_value_1": "", "auth_value_2": "", "auth_value_3": "", "auth_endpoint": "https://login.microsoftonline.com/%auth_value_1%/oauth2/v2.0/token", "auth_data_node": "access_token" }, "connector_data": { "id": "AZURE_AD", "display_name": "Azure Active Directory", "display_description": "Connector for processing Azure Active Directory data using MS Graph.", "enabled": "1", "help_url": "/wikis/platform-user-guide/connector-azure-ad", "connection_type": "both", "connector_version": "2024-08-31", "authentications": [ { "id": "AZURE_AD_ms_graph", "display_name": "MS Graph Authentication", "display_description": "Connect to an Azure Active Directory using MS Graph API", "enabled": "1", "fields": [ { "field_name": "source_server", "field_label": "API Domain", "default_value": "https://graph.microsoft.com" }, { "field_name": "data_object", "field_label": "Server Temporary Working Path", "default_value": "storage/etl5" }, { "field_name": "auth_value_1", "field_label": "Tenant Id", "default_value": "" }, { "field_name": "auth_value_2", "field_label": "App ID / Client ID", "default_value": "" }, { "field_name": "auth_value_3", "field_label": "Client Secret", "default_value": "" }, { "field_name": "auth_endpoint", "field_label": "OAuth POST Request Full URI", "default_value": "https://login.microsoftonline.com/%auth_value_1%/oauth2/v2.0/token" }, { "field_name": "auth_data_node", "field_label": "Access Token JSON Node", "default_value": "access_token" } ] } ], "job_templates": [ { "job_description": "Users", "connection_type": "inbound", "schedule": "Daily" }, { "job_description": "Computers", "connection_type": "inbound", "schedule": "Daily" }, { "job_description": "Groups", "connection_type": "inbound", "schedule": "Daily" }, { "job_description": "Group Members", "connection_type": "inbound", "schedule": "Daily" }, { "job_description": "Group Owners", "connection_type": "inbound", "schedule": "Daily" } ] } }

Headers

Name	Type
Date	string
Server	string
X-Powered-By	string
Content-Length	string
Keep-Alive	string
Connection	string

etl/activateConnection/{authentication_id} Activates a new connection for a specific connector. The endpoint requires an authentication_id variable.

It also requires a JSON Body Payload with the activation data.

Payload Example:

{
    "label": "connection_name",
    "auth_value_1": "authentication value 1",
    "auth_value_2": "authentication value 2",
    "auth_value_3": "authentication value 3",
    "enable_jobs": true
}

URL:https://webroot/api/v1/etl/activateConnection/{authentication_id}

authentication_id = String

Example:https://webroot/api/v1/etl/activateConnection/AZURE_AD_ms_graph

Required Fields:

To use the default value for a defined field, omit the field from the body payload.

Setting the optional enable_jobs field to true will enable all the connection jobs if they are disabled in the connector job templates.

Parameters

Name	Description
Authorization string (header) * required	Default value: Bearer {token}
Body (body)	Example Value
	application/json
	{ "label": "connection_name", "auth_value_1": "authentication value 1", "auth_value_2": "authentication value 2", "auth_value_3": "authentication value 3", "enable_jobs": true }

Responses

Code	Description
200	Example Value
	application/json
	{ "message": "Connection [connection_name] activated successfully.", "connection_id": "999" }

Headers

Name	Type
Date	string
Server	string
X-Powered-By	string
Content-Length	string
Keep-Alive	string
Connection	string

etl/connections Returns a list of all activated ETL connections.

URL:https://webroot/api/v1/etl/connections

Parameters

Name	Description
Authorization string (header) * required	Default value: Bearer {token}

Responses

Code	Description
200	Example Value
	application/json
	{ "JSON Rest API": { "connector_id": "JSON Rest API", "connector_name": "Generic Rest JSON API", "connections": [ { "connection_id": "41", "connection_label": "readyworks", "authentication_type": "Bearer Token", "connection_jobs": { "inbound_jobs": [ { "connection_job_id": "275", "job_description": "Get Users", "job_active": "0" }, { "connection_job_id": "273", "job_description": "Get Computers", "job_active": "0" } ], "outbound_jobs": [ { "connection_job_id": "274", "job_description": "POST Service Incident", "job_active": "0" } ] } } ] }, "INTUNE": { "connector_id": "INTUNE", "connector_name": "Microsoft Intune", "connections": [ { "connection_id": "26", "connection_label": "intune", "authentication_type": "MS Graph Authentication", "connection_jobs": { "inbound_jobs": [ { "connection_job_id": "175", "job_description": "Users", "job_active": "0" }, { "connection_job_id": "177", "job_description": "User Registered Devices", "job_active": "0" }, { "connection_job_id": "178", "job_description": "Groups", "job_active": "0" }, { "connection_job_id": "179", "job_description": "Group Members", "job_active": "0" }, { "connection_job_id": "180", "job_description": "Detected Apps", "job_active": "0" }, { "connection_job_id": "181", "job_description": "Managed Applications", "job_active": "0" }, { "connection_job_id": "182", "job_description": "Managed Applications Assignments", "job_active": "0" }, { "connection_job_id": "183", "job_description": "Managed Devices", "job_active": "0" }, { "connection_job_id": "184", "job_description": "Managed Devices Hardware", "job_active": "0" }, { "connection_job_id": "176", "job_description": "Protection State", "job_active": "0" } ] } } ] }, "SNOW": { "connector_id": "SNOW", "connector_name": "ServiceNow (SNOW)", "connections": [ { "connection_id": "44", "connection_label": "servicenow", "authentication_type": "Username and Password", "connection_jobs": { "inbound_jobs": [ { "connection_job_id": "300", "job_description": "User Group", "job_active": "0" }, { "connection_job_id": "299", "job_description": "User", "job_active": "0" }, { "connection_job_id": "298", "job_description": "Phone Territory", "job_active": "0" }, { "connection_job_id": "297", "job_description": "Company", "job_active": "0" }, { "connection_job_id": "296", "job_description": "Location", "job_active": "0" }, { "connection_job_id": "295", "job_description": "Department", "job_active": "0" }, { "connection_job_id": "294", "job_description": "Cost Center", "job_active": "0" }, { "connection_job_id": "293", "job_description": "Device Model Category", "job_active": "0" }, { "connection_job_id": "292", "job_description": "Device Model", "job_active": "0" }, { "connection_job_id": "291", "job_description": "Depreciation", "job_active": "0" }, { "connection_job_id": "290", "job_description": "CMDB Configuration", "job_active": "0" }, { "connection_job_id": "289", "job_description": "Device", "job_active": "0" } ] } } ] }, "AZURE_AD": { "connector_id": "AZURE_AD", "connector_name": "Azure Active Directory", "connections": [ { "connection_id": "23", "connection_label": "azure_ad", "authentication_type": "MS Graph Authentication", "connection_jobs": { "inbound_jobs": [ { "connection_job_id": "164", "job_description": "Users", "job_active": "1" }, { "connection_job_id": "165", "job_description": "Computers", "job_active": "1" }, { "connection_job_id": "166", "job_description": "Groups", "job_active": "1" }, { "connection_job_id": "167", "job_description": "Group Members", "job_active": "1" }, { "connection_job_id": "168", "job_description": "Group Owners", "job_active": "1" } ], "outbound_jobs": [ { "connection_job_id": "169", "job_description": "Create Group", "job_active": "1" }, { "connection_job_id": "170", "job_description": "Delete Group", "job_active": "1" }, { "connection_job_id": "171", "job_description": "Update Group", "job_active": "1" } ] } } ] } }

Headers

Name	Type
Date	string
Server	string
X-Powered-By	string
Content-Length	string
Keep-Alive	string
Connection	string

etl/connectors/{needle} Search connections using the needle. Returns an empty set if the needle is not found.

URL:https://webroot/api/v1/etl/connections/{needle}

needle = String

Example:https://webroot/api/v1/etl/connections/azure

Parameters

Name	Description
Authorization string (header) * required	Default value: Bearer {token}

Responses

Code	Description
200	Example Value
	application/json
	{ "AZURE_AD": { "connector_id": "AZURE_AD", "connector_name": "Azure Active Directory", "connections": [ { "connection_id": "23", "connection_label": "azure_ad", "authentication_type": "MS Graph Authentication", "connection_jobs": { "inbound_jobs": [ { "connection_job_id": "164", "job_description": "Users", "job_active": "1" }, { "connection_job_id": "165", "job_description": "Computers", "job_active": "1" }, { "connection_job_id": "166", "job_description": "Groups", "job_active": "1" }, { "connection_job_id": "167", "job_description": "Group Members", "job_active": "1" }, { "connection_job_id": "168", "job_description": "Group Owners", "job_active": "1" } ], "outbound_jobs": [ { "connection_job_id": "169", "job_description": "Create Group", "job_active": "1" }, { "connection_job_id": "170", "job_description": "Delete Group", "job_active": "1" }, { "connection_job_id": "171", "job_description": "Update Group", "job_active": "1" } ] } } ] } }

Headers

Name	Type
Date	string
Server	string
X-Powered-By	string
Content-Length	string
Keep-Alive	string
Connection	string

etl/connection/{connection_id} Returns information on a specific connection. The endpoint requires a connection_id variable.

URL:https://webroot/api/v1/etl/connection/{connection_id}

connection_id = String

Example:https://webroot/api/v1/etl/connection/23

You can find the connection_id by using the etl/connections endpoint:

{
    "AZURE_AD": {
        "connector_id": "AZURE_AD",
        "connector_name": "Azure Active Directory",
        "connections": [
            {
                "connection_id": "23",
                "connection_label": "azure_ad",
                "authentication_type": "MS Graph Authentication",
                "connection_jobs": {
                    "inbound_jobs": [
                        {
                            "connection_job_id": "164",
                            "job_description": "Users",
                            "job_active": "1"
                        },
                        {
                            "connection_job_id": "165",
                            "job_description": "Computers",
                            "job_active": "1"
                        },
                        {
                            "connection_job_id": "166",
                            "job_description": "Groups",
                            "job_active": "1"
                        },
                        {
                            "connection_job_id": "167",
                            "job_description": "Group Members",
                            "job_active": "1"
                        },
                        {
                            "connection_job_id": "168",
                            "job_description": "Group Owners",
                            "job_active": "1"
                        }
                    ],
                    "outbound_jobs": [
                        {
                            "connection_job_id": "169",
                            "job_description": "Create Group",
                            "job_active": "1"
                        },
                        {
                            "connection_job_id": "170",
                            "job_description": "Delete Group",
                            "job_active": "1"
                        },
                        {
                            "connection_job_id": "171",
                            "job_description": "Update Group",
                            "job_active": "1"
                        }
                    ]
                }
            }
        ]
    }
}

Parameters

Name	Description
Authorization string (header)	Default value: Bearer {token}

Responses

Code	Description
200	Example Value
	application/json
	{ "AZURE_AD": { "connector_id": "AZURE_AD", "connector_name": "Azure Active Directory", "connections": [ { "connection_id": "23", "connection_label": "ad", "authentication_type": "MS Graph Authentication", "connection_jobs": { "inbound_jobs": [ { "connection_job_id": "164", "job_description": "Users", "job_active": "1" }, { "connection_job_id": "165", "job_description": "Computers", "job_active": "1" }, { "connection_job_id": "166", "job_description": "Groups", "job_active": "1" }, { "connection_job_id": "167", "job_description": "Group Members", "job_active": "1" }, { "connection_job_id": "168", "job_description": "Group Owners", "job_active": "1" } ], "outbound_jobs": [ { "connection_job_id": "169", "job_description": "Create Group", "job_active": "1" }, { "connection_job_id": "170", "job_description": "Delete Group", "job_active": "1" }, { "connection_job_id": "171", "job_description": "Update Group", "job_active": "1" } ] } } ] } }

Headers

Name	Type
Date	string
Server	string
X-Powered-By	string
Content-Length	string
Keep-Alive	string
Connection	string

Table of Contents

Parameters

Responses

Headers

Parameters

Responses

Headers

Parameters

Responses

Headers

Parameters

Responses

Headers

Parameters

Responses

Headers

Parameters

Responses

Headers

Parameters

Responses

Headers

Attachments