PowerTrack Rules API

Methods ¶

Method	Description
POST /rules	Add rules to the stream
GET /rules	Retrieve all rules currently in place on the stream
GET /rules/:rule_id	Retrieve an existing rule on the stream by rule ID
POST /rules _method=get	Retrieve multiple rules on the stream by rule IDs
POST /rules _method=delete	Delete rules from the stream
POST /validation	Validate PowerTrack rule syntax

Authentication ¶

All requests to the PowerTrack rules API must use HTTP Basic Authentication, constructed from a valid email address and password combination used to log into your account at console.gnip.com. Credentials must be passed as the Authorization header for each request. Make sure your client is adding the "Authentication: Basic" HTTP header (with encoded credentials over HTTPS) to all API requests.

POST /rules ¶

Adds one or many rules to your PowerTrack stream's ruleset.

Request Specifications

Request Method	HTTP POST
Content Type	"application/json". The request should specify this as the "Content-type".
URL	Found on the Console - Products API Help tab, and uses the following structure: https://data-api.x.com/rules/powertrack/accounts/{gnip_account_name}/publishers/twitter/{stream_label}.json
Character Encoding	UTF-8
Request Body Format	JSON
Request Body Size Limit	5 MB
Rate Limit	60 requests per minute, aggregated across all requests to /rules endpoint for the specific stream's API (POST and GET). Rule addition requests will be processed serially and will be rejected if you have more than one rule request happening at the same time.

Request Body Content

Your request should provide rule data in the following format with the defined Content-type: "application/json":

{
    "rules":
    [
        {"value":"rule1","tag":"tag1"},
        {"value":"rule2","tag":"tag2"}
    ]
}

Example curl Request

The following example requests demonstrate how to add rules using cURL on the command line, using JSON rules.

curl -v -X POST -uexample@customer.com "https://data-api.x.com/rules/powertrack/accounts/:account_name/publishers/twitter/:stream_label.json" -d '{"rules":[{"value":"rule1","tag":"tag1"},{"value":"rule2","tag":"tag2"}]}'

curl -v -X POST -uexample@customer.com "https://data-api.x.com/rules/powertrack/accounts/:account_name/publishers/twitter/:stream_label.json" -d @arrayofrulesfile_max5mb.json

curl -v -X POST -uexample@customer.com "https://data-api.x.com/rules/powertrack/accounts/companyname/publishers/twitter/prod.json" -d '{"rules":[{"value":"(#snow OR snowday OR "snow day" OR from:noaa) lang:en","tag":"4581245"},{"value":"(skiing OR boarding OR "hitting the slopes" OR from:OnTheSnow) lang:en","tag":"4581267"}]}'

Responses

The following responses may be returned by the API for these requests. Non-200 responses should be retried after making any necessary modifications in the rules.

Status	Text	Description
201	Created	The rule or rules were successfully added to your PowerTrack ruleset.
400	Bad Request	Generally relates to poorly formatted JSON, and includes an "Invalid JSON" message in the response.
401	Unauthorized	HTTP authentication failed due to invalid credentials. Log in to console.gnip.com with your credentials to ensure you are using them correctly with your request.
422	Unprocessable Entity	Generally occurs due to an invalid rule, based on the PowerTrack rule restrictions. Requests fail or succeed as a batch. For these errors, each invalid rule and the reason for rejection is included in a JSON message in the response. Catch the associated exception to expose this message.
429	Rate Limited	Your app has exceeded the limit on requests to add, delete, or list rules for this stream.
503	Service Unavailable	Twitter server issue. Reconnect using an exponential backoff pattern. If no notice about this issue has been posted on the Twitter API Status Page contact support or contact emergency if still unable to connect after 10 minutes.

Example Responses

This response indicates that all rules (two in this case) were successfully created.

HTTP/1.1 201 Created
{
    "summary": {
        "created": 2,
        "not_created": 0
    },
    "detail": [{
        "rule": {
            "value": "(#snow OR snowday OR \"snow day\" OR from:noaa) lang:en",
            "tag": "4581245",
            "id": 734938587381145604
        },
        "created": true
    }, {
        "rule": {
            "value": "(skiing OR boarding OR \"hitting the slopes\" OR from:OnTheSnow) lang:en",
            "tag": "4581267",
            "id": 734938587381174273
        },
        "created": true
    }],
    "sent": "2016-05-24T02:46:28.402Z"
}

This response indicates that one rule was successfully created, and two were not created because they already exist. Rules are indexed by rule value (syntax). For rules not created there is a 'message' field explaining why the rule could not be created.

HTTP/1.1 201 Created
{
    "summary": {
        "created": 1,
        "not_created": 2
    },
    "detail": [{
        "rule": {
            "value": "robot OR 🤖",
            "tag": "botrule123",
            "id": 734861971116285952
        },
        "created": true
    }, {
        "rule": {
            "value": "fish OR 🐟",
            "tag": "fishrule123"
        },
        "created": false,
        "message": "A rule with this value already exists"
    }, {
        "rule": {
            "value": "Pizza OR 🍕",
            "tag": "pizzarule123"
        },
        "created": false,
        "message": "A rule with this value already exists"
    }],
    "sent": "2016-05-23T21:42:01.661Z"
}

The following responses indicate that no rules were created. In each case there is a 'message' field explaining why the rule could not be created. Note that when one or more rules are invalid, no rules are added (even rules with valid syntax).

HTTP/1.1 422 Unprocessable Entity
{
    "summary": {
        "created": 0,
        "not_created": 2
    },
    "detail": [{
        "rule": {
            "value": "streaming gnip contains:$twtr",
            "tag": null
        },
        "created": false,
        "message": "no viable alternative at input '$twtr' (at position 25)\n"
    }, {
        "rule": {
            "value": "streaming gnip contains:\"$twtr\"",
            "tag": null
        },
        "created": false
    }],
    "sent": "2016-06-01T15:41:27.451Z"
}

HTTP/1.1 422 Unprocessable Entity
{
    "summary": {
        "created": 0,
        "not_created": 1
    },
    "detail": [{
        "rule": {
            "value": "-follow",
            "tag": null
        },
        "created": false,
        "message": "Rules must contain a non-negation term (at position 1)\nRules must contain at least one positive, non-stopword clause (at position 1)\n"
    }],
    "sent": "2016-05-23T18:34:25.218Z"
}

HTTP/1.1 422 Unprocessable Entity
{
    "summary": {
        "created": 0,
        "not_created": 1
    },
    "detail": [{
        "rule": {
            "value": "streaming AND lang:en",
            "tag": null
        },
        "created": false,
        "message": "Ambiguous use of and as a keyword. Use a space to logically join two clauses, or \"and\" to find occurrences of and in text (at position 11)\n"
    }],
    "sent": "2016-05-23T21:39:49.612Z"
}

GET /rules ¶

Retrieve all rules currently in place on the stream

Request Specifications

Request Method	HTTP GET
URL	Found on the Console - Products API Help tab, and uses the following structure: https://data-api.x.com/rules/powertrack/accounts/:account_name/publishers/twitter/:stream_label.json
Rate Limit	60 requests per minute, aggregated across all requests to /rules endpoint for the specific stream's API (POST and GET).

Example cURL Request

The following example request demonstrates how to retrieve rules using cURL on the command line.

curl -v -uexample@customer.com "https://data-api.x.com/rules/powertrack/accounts/:account_name/publishers/twitter/:stream_label.json"

Response

The following responses may be returned by the API for these requests. Non-200 responses should be retried, utilizing an exponential backoff for subsequent requests.

Status	Text	Description
200	OK	The request was successful, and the current ruleset is returned in JSON format.
400	Bad Request	Generally relates to poorly formatted JSON, and includes an "Invalid JSON" message in the response.
401	Unauthorized	HTTP authentication failed due to invalid credentials. Log in to console.gnip.com with your credentials to ensure you are using them correctly with your request.
429	Rate Limited	Your app has exceeded the limit on requests to add, delete, or list rules for this stream.
503	Service Unavailable	Twitter server issue. Reconnect using an exponential backoff pattern. If no notice about this issue has been posted on the Twitter API Status Page contact support.

Example Response

HTTP/1.1 200 OK
{
    "rules": [{
        "value": "from:twitterdev",
        "tag": "tweetsfromtwitterdev123",
        "id": 735163830813134848
    }, {
        "value": "fish OR 🐟",
        "tag": "fishrule123",
        "id": 738034522583769088
    }, {
        "value": "Pizza OR 🍕",
        "tag": "pizzarule123",
        "id": 738034522579554304
    }, {
        "value": "robot OR 🤖",
        "tag": "botrule123",
        "id": 738034522579570689
    }],
    "sent": "2016-06-01T15:52:37.341Z"
}

GET /rules/:rule_id ¶

Retrieve an existing rule on the stream by rule ID. Note that all rules are assigned a unique ID by Twitter at the time of creation, rules that are deleted and recreated will receive a different unique rule ID.

Request Specifications

Request Method	HTTP GET
URL	Found on the Console - Products API Help tab, and uses the following structure: https://data-api.x.com/rules/powertrack/accounts/:account_name/publishers/twitter/:stream_label/rules/:rule_id.json
Rate Limit	60 requests per minute, aggregated across all requests to /rules endpoint for the specific stream's API (POST and GET).

Example cURL Request

The following example request demonstrates how to retrieve a rule by rule_id using cURL on the command line.

curl -v -uexample@customer.com "https://data-api.x.com/rules/powertrack/accounts/:account_name/publishers/twitter/:stream_label/rules/:rule_id.json"

curl -v -uexample@customer.com "https://data-api.x.com/rules/powertrack/accounts/companyname/publishers/twitter/prod/rules/735163830813134848.json"

Response

The following responses may be returned by the API for these requests. Non-200 responses should be retried, utilizing an exponential backoff for subsequent requests.

Status	Text	Description
200	OK	The request was successful, and the current rule is returned in JSON format.
400	Bad Request	Generally relates to poorly formatted JSON, and includes an "Invalid JSON" message in the response.
401	Unauthorized	HTTP authentication failed due to invalid credentials. Log in to console.gnip.com with your credentials to ensure you are using them correctly with your request.
429	Rate Limited	Your app has exceeded the limit on requests to add, delete, or list rules for this stream.
503	Service Unavailable	Twitter server issue. Reconnect using an exponential backoff pattern. If no notice about this issue has been posted on the Twitter API Status Page contact support.

Example Response

HTTP/1.1 200 OK
{
    "rules": [{
        "value": "from:twitterdev",
        "tag": "tweetsfromtwitterdev123",
        "id": 735163830813134848
        "id_str":"735163830813134848"
    }],
    "sent": "2016-06-01T15:52:37.341Z"
}

POST /rules _method=get ¶

Retrieves requested existing rules by list of rule IDs currently on a stream.

Request Specifications

Request Method	HTTP POST
URL	Found on the API Help page, and uses the following structure: https://data-api.x.com/rules/powertrack/accounts/{gnip_account_name}/publishers/twitter/{stream_label}.json?_method=get
Character Encoding	UTF-8
Request Body Format	JSON
Request Body Size Limit	5 MB
Rate Limit	60 requests per minute, aggregated across all requests to /rules endpoint for the specific stream's API (POST and GET).
Compression	Gzip compression is supported, but not required for these requests.

Example curl Request

The following example request demonstrates how to add rules using cURL on the command line.

curl -v -X POST -uexample@customer.com \
"https://data-api.x.com/rules/powertrack/accounts/:account_name/publishers/twitter/:stream_label.json?_method=get" \
-d '{"rule_ids":[734938587381145604,734938587381174273]}"

Response

The following responses may be returned by the API for these requests. Non-200 responses should be retried, utilizing an exponential backoff for subsequent requests.

Status	Text	Description
200	OK	The request was successful, and the current ruleset is returned in JSON format.
400	Bad Request	Generally relates to poorly formatted JSON, and includes an "Invalid JSON" message in the response.
401	Unauthorized	HTTP authentication failed due to invalid credentials. Log in to console.gnip.com with your credentials to ensure you are using them correctly with your request.
429	Rate Limited	Your app has exceeded the limit on requests to add, delete, or list rules for this stream.
503	Service Unavailable	Twitter server issue. Reconnect using an exponential backoff pattern. If no notice about this issue has been posted on the Twitter API Status Page contact support.

Example Response

HTTP/1.1 200 OK
{
    "rules": [{
        "value": "from:furiouscamper",
        "tag": null,
        "id": 734938587381145604
    }, {
        "value": "fish 🐟",
        "tag": null,
        "id": 734938587381174273
    }],
    "sent": "2016-06-01T15:52:37.341Z"
}

    {
    "error": {
        "message": "Invalid JSON. The body must be in the format {\"rules\":[{\"value\":\"rule1\", \"tag\":\"tag1\"}, {\"value\":\"rule2\"}]} or {\"rule_ids\": [rule_id1, rule_id2, rule_id3, rule_id4, rule_id5]}",
        "sent": "2013-08-16T00:50:00+00:00"
    }
}

POST /rules _method=delete ¶

Deletes requested existing rules by list of rule values or rule IDs currently on a stream.

Request Specifications

Request Method	HTTP POST
Authentication	Basic Authentication. Your login credentials must be included in the header of the request.
Content Type	"application/json". The request should specify this as the "Content-type".
URL	Found on the API Help page, and uses the following structure: https://data-api.x.com/rules/powertrack/accounts/{gnip_account_name}/publishers/twitter/{stream_label}.json?_method=delete
Character Encoding	UTF-8
Request Body Format	JSON
Request Body Size Limit	5 MB
Rate Limit	60 requests per minute, aggregated across all requests to /rules endpoint for the specific stream's API (POST and GET).

Request Body Content

Your request should provide rule data in the following formats:

Content-type: "application/json"
{
    "rules":
    [
        {"value":"rule1"},
        {"value":"rule2"}
    ]
}

Content-type: "application/json"
{
    "rule_ids":
    [
        734938587381145604,
        734938587381174273
    ]
}

Example curl Request

The following example request demonstrates how to add rules using cURL on the command line.

curl -v -X POST -uexample@customer.com \
"https://data-api.x.com/rules/powertrack/accounts/:account_name/publishers/twitter/:stream_label.json?_method=delete" \
-d '{"rules":[{"value":"testrule"}]}"

curl -v -X POST -uexample@customer.com \
"https://data-api.x.com/rules/powertrack/accounts/:account_name/publishers/twitter/:stream_label.json?_method=delete" \
-d '{"rule_ids":[734938587381145604,734938587381174273]}"

Responses

The following responses may be returned by the API for these requests. Non-200 responses should be retried following any necessary modifications to the rules being deleted.

Status	Text	Description
200	OK	Indicates that the rule data supplied with the request consisted of valid JSON. However, note that if no rules are found in the ruleset for the PowerTrack stream based on a case-sensitive search, no rules will be deleted.
400	Bad Request	Generally relates to poorly formatted JSON, and includes an "Invalid JSON" message in the response.
401	Unauthorized	HTTP authentication failed due to invalid credentials. Log in to console.gnip.com with your credentials to ensure you are using them correctly with your request.
429	Rate Limited	Your app has exceeded the limit on requests to add, delete, or list rules for this stream.
503	Service Unavailable	Twitter server issue. Reconnect using an exponential backoff pattern. If no notice about this issue has been posted on the Twitter API Status Page, contact support.

Example Responses

HTTP/1.1 200 OK
{
    "summary": {
        "deleted": 3,
        "not_deleted": 0
    },
    "detail": [],
    "sent": "2016-06-01T15:46:48.654Z"
}

HTTP/1.1 200 OK
{
    "summary": {
        "deleted": 0,
        "not_deleted": 3
    },
    "detail": [{
        "rule": {
            "value": "Pizza",
            "tag": null
        },
        "deleted": false,
        "message": "Rule does not exist"
    }, {
        "rule": {
            "value": "eggplant",
            "tag": null
        },
        "deleted": false,
        "message": "Rule does not exist"
    }, {
        "rule": {
            "value": "fish",
            "tag": null
        },
        "deleted": false,
        "message": "Rule does not exist"
    }],
    "sent": "2016-06-01T15:49:15.951Z"
}

HTTP/1.1 400 Bad Request
{
    "error": {
        "message": "Invalid JSON. The body must be in the format {\"rules\":[{\"value\":\"rule1\", \"tag\":\"tag1\"}, {\"value\":\"rule2\"}]} or {\"rule_ids\": [rule_id1, rule_id2, rule_id3, rule_id4, rule_id5]}",
        "sent": "2013-08-16T00:50:00+00:00"
    }
}

Important note on rule management: Rule sets are indexed by the value or ruleID, not the tag; therefore, all rule additions must reference the rule value or ruleID. In order to to make a tag update to an existing rule, you must first delete it and then add it back with the new tag value.

Rules must be unique per stream based on rule value, see below for a rule management example scenario:

CREATE RULE

Action: POST rule {"value":"#TwitterData","tag":"tagtextA"} {"summary":{"created":1,"not_created":0},"detail":[{"rule":{"value":"#TwitterData","tag":"tagtextA","id":961664522481119232,"id_str":"961664522481119232"},"created":true}],"sent":"2018-02-08T18:14:23.691Z"} System: {"value":"#TwitterData","tag":"tagtextA","id":961664522481119232,"id_str":"961664522481119232"}

FAILED ATTEMPT TO UPDATE TAG

Action: POST rule {"value":"#TwitterData","tag":"tagtextB"} **Rule tags cannot be "updated" - This request is ignored because rule value #TwitterData already exists. {"summary":{"created":0,"not_created":1},"detail":[{"rule":{"value":"#TwitterData","tag":"tagtextB","id":961664522481119232,"id_str":"961664522481119232"},"created":false,"message":"A rule with this value already exists"} System: {"value":"#TwitterData","tag":"tagtextA","id":961664522481119232,"id_str":"961664522481119232"}

FAILED ATTEMPT TO DELETE BY TAG

Action: POST method=delete rule {"tag":"tagtextA"} **Rules cannot be deleted by tag. {"summary":{"deleted":0,"not_deleted":1},"detail":[{"rule":{"value":"","tag":null},"deleted":false,"message":"Rule does not exist"}],"sent":"2018-02-08T18:42:37.004Z"} System: {"value":"#TwitterData","tag":"tagtextA","id":961664522481119232,"id_str":"961664522481119232"}

DELETE BY ID

Action: POST method=delete rule {"rule_ids":[961664522481119232]} {"summary":{"deleted":1,"not_deleted":0},"detail":[],"sent":"2018-02-08T18:53:54.185Z"}

DELETE BY VALUE

Action: POST method=delete rule {"value":"#TwitterData"} {"summary":{"deleted":1,"not_deleted":0},"detail":[],"sent":"2018-02-08T18:53:54.185Z"}

RECREATE RULE- NOW WITH NEW ID

Action: POST rule {"value":"#TwitterData","tag":"tagtextB"} {"summary":{"created":1,"not_created":0},"detail":[{"rule":{"value":"#TwitterData","tag":"tagtextB","id":961675641140609025,"id_str":"961675641140609025"},"created":true}],"sent":"2018-02-08T18:58:34.586Z"} System: {"value":"#TwitterData","tag":"tagtextB","id":961675641140609025,"id_str":"961675641140609025"}

POST /validation ¶

Validates PowerTrack rules.

Note: Using this endpoint will not impact your PowerTrack streams.

Request Specifications

Request Method	HTTP POST
URL	Found on the API Help page in console, and uses the following structure: https://data-api.x.com/rules/powertrack/accounts/:account_name/publishers/twitter/:stream_label/validation.json
Character Encoding	UTF-8
Request Body Format	JSON
Request Body Size Limit	5 MB
Rate Limit	60 requests per minute, aggregated across all requests to /rules endpoint for the specific stream's API (POST and GET).

Example curl Request

The following example request demonstrates how to add rules using cURL on the command line.

curl --compressed -v -uexample@customer.com \
"https://data-api.x.com/rules/powertrack/accounts/:account_name/publishers/twitter/:stream_label/validation.json" \
-d '{
    "rules": [{
        "value": "Pizza OR 🍕 OR \"🍕\" sample:100"
    }, {
        "value": "from:contains:heart"
    }, {
        "value": "fish AND bird"
    }, {
        "value": "(((\"#quotedhashtag\""
    }, {
        "value": "bounding_box:[-71.199636,42.230046,-70.979909,42.398619]"
    }, {
        "value": "from:jack"
    }]
}'

Response

The following responses may be returned by the API for these requests. Non-200 responses should be retried, utilizing an exponential backoff for subsequent requests.

Status	Text	Description
200	OK	The request was successful, and the rule validation result is returned.
400	Bad Request	Generally relates to poorly formatted JSON, and includes an "Invalid JSON" message in the response.
401	Unauthorized	HTTP authentication failed due to invalid credentials. Log in to console.gnip.com with your credentials to ensure you are using them correctly with your request.
429	Rate Limited	Your app has exceeded the limit on requests to add, delete, or list rules for this stream.
503	Service Unavailable	Twitter server issue. Reconnect using an exponential backoff pattern. If no notice about this issue has been posted on the Twitter API Status Page, contact support.

Example Response

This response indicates that one rule is valid and five are not valid. For rules that are not valid, there is a 'message' field explaining why the rule is not valid.

HTTP/1.1 200 OK
{
    "summary": {
        "valid": 1,
        "not_valid": 5
    },
    "detail": [{
        "rule": {
            "value": "from:jack",
            "tag": null
        },
        "valid": true
    }, {
        "rule": {
            "value": "Pizza OR 🍕 OR \"🍕\" sample:100",
            "tag": null
        },
        "valid": false,
        "message": "The sample operator cannot be used with an OR. To use the sample operator with an OR in the rule, the ORed clauses must be grouped together with parenthesis.  For example, to get 10% of activites that have term1 or term2, the rule should be (excluding the single quotes) '(term1 OR term2) sample:10' (at position 21)\n"
    }, {
        "rule": {
            "value": "from:contains:heart",
            "tag": null
        },
        "valid": false,
        "message": "Cannot parse rule at ':' (position 14)\n"
    }, {
        "rule": {
            "value": "fish AND bird",
            "tag": null
        },
        "valid": false,
        "message": "Ambiguous use of and as a keyword. Use a space to logically join two clauses, or \"and\" to find occurrences of and in text (at position 6)\n"
    }, {
        "rule": {
            "value": "(((\"#quotedhashtag\"",
            "tag": null
        },
        "valid": false,
        "message": "mismatched input 'EOF' expecting ')' (at position 20)\n\n"
    }, {
        "rule": {
            "value": "bounding_box:[-71.199636,42.230046,-70.979909,42.398619]",
            "tag": null
        },
        "valid": false,
        "message": "Cannot parse rule at '71.199636,42.230046,-70.979909,42.398619' (position 16)\n"
    }],
    "sent": "2017-03-16T02:33:01.827Z"
}