Learning path / How to detect signal from noise and build powerful filtering rules
Step 1: Understanding the basics of rules and query operators
This is step 1 of the learning path, How to detect signal from noise and build powerful filtering rules.
Before you start creating filters, it’s important to understand some of the basics behind rules syntax and logic, as well as some of the nuances around how the filtering operators work and how rules are evaluated logically.
Rules are made up of one or more “clauses,” where a clause is a keyword, exact phrase, or one of many enterprise filtering operators. Multiple clauses can be combined using boolean syntax to form a filtering rule.
| Boolean syntax | Operator | Example |
|---|---|---|
| AND | Use whitespace to imply "AND" logic. Please note: Do not use “AND” explicitly in your rule. Only use whitespace. An explicit “AND” will be treated like a regular keyword. |
apple mac |
| OR | Use an uppercase OR | apple OR mac |
| NOT | Use a hyphen to negate terms or clauses. | apple -fruit |
The above logic can be combined using groupings to expand your filtering rules into more complex queries.
| Groupings | Example |
Order of operators: "AND" logic always takes precedence over "OR" logic.
|
ipad OR apple mac The above rule will be evaluated as follows: In other words, the above rule will match Tweets that either contain both the words “apple” and “mac” or the word “ipad.” |
Parentheses can be used to form functional groupings. Use parentheses to create groupings and override the order of operators described above. Parentheses dictate in what order the different clauses should be evaluated. |
(ipad OR apple) mac The above rule is the same as: In other words, the above rule will match Tweets that contain the words “mac” and “ipad,” or the words “mac” and “apple.” |
More than 50 filtering operators are available to use with our real-time filtering API at the enterprise level (in other words, the PowerTrack API). Broadly speaking, these operators can be categorized as follows.
Please note:
The operators outlined in bold below are only available with the PowerTrack API. All other operators are available with both the PowerTrack and Search APIs. You can find a list of operators by product in our documentation.
| Matching on Tweet content | Matching on accounts of interest | Matching on Tweet attributes or Tweet type | Matching on geospatial operators |
| keyword | from: | has:links | point_radius: |
| emoji | to: | sample: | bounding_box: |
| “exact phrase match” | @ | source: | bio_location: |
| “keyword1 keyword2” ~N (proximity operator) | bio: | has:lang | place: |
| contains: | bio_name: | has:hashtags | place_country: |
| url: | retweets_of: | has:images | has:geo |
| url_title: | statuses_count: | has:videos | has:profile_geo |
| url_description: | followers_count: | has:media | profile_point_radius: |
| url_contains: | friends_count: | has:mentions | profile_bounding_box: |
| # | listed_count: | has:symbols | profile_country: |
| $ | is:verified | is:retweet | profile_region: |
| lang: | is:reply | profile_locality: | |
| is:quote | profile_subregion: | ||
| retweets_of_status_id: | |||
| Note: keywords and “exact phrase matches” are not case sensitive. | in_reply_to_status_id: | ||
| -is:nullcast Note: this operator can only be used in its negated form. |
When building rules, you can mix operators from the different categories outlined above, and there are no limits as to how many times an operator can be used. With our enterprise products, each rule can be up to 2,048 characters long, with no limits on the number of positive clauses and negative clauses.
Please note:
Every character in a rule string counts towards the 2,048 character limit. This includes the operators themselves (for example, from:), the colon character, as well as boolean syntax (for example, OR).
The following is a visual representation of the different types of filtering operators:
A full list of available enterprise operators and associated descriptions can be found in our documentation here.
As you start building your rule(s), note that a typical rule pattern might look something like this:
(keyword1 OR keyword2 OR keyword3) (#hashtag1 OR #hashtag2) -is:retweet ↑ ↑ ↑ Grouping 1 Grouping 2 Negation(s) |
Go to the next article - Step 2: Identifying what “signal” means to you
Go back to the learning path homepage - Learning path: How to detect signal from noise and build powerful filtering rules