To use DataOps REST API In the ADF pipeline we need to add 2 web activities from the General section.

for example, if we add two activities with the names Web1 and Web2

for  Web1 activity, we need to use the below settings(This is to get a token, we will use this token in Web2 using JSON output)

URL: https://<host>/dataopssecurity/oauth/token

Method:POST

Headers:  Name: Content-Type    Value: application/x-www-form-urlencoded

Body:username=<username>&password=<encrypted password>%3D&grant_type=password

! need to replace <username> and <encrypted password>(you can get it from the user profile screen)

Authentication: Basic

Username: <Client ID need to get it from datagaps Team>

Password: <Client Secret need to get it from datagaps Team>

for Web2 activity, we need to use the below settings

URL: https://<host>/DataFlowService/api/v1.0/dataFlows/executeDataFlow?dataflowId=<dataflowID>&livyid=<Livy Server ID>

You can get <dataflowID> from the dataflow edit screen and <Livy Server ID> from the cluster screen

Method: POST

Headers: Name: Authorization  Value:@{concat('Bearer ', activity('Web1').output.access_token)}