Decoding Hate

Following the outbreak of COVID-19, there was a rise in hate incidents directed towards Asian Americans and Pacific Islanders (AAPIs). While some of these incidents were covered in the news, thousands of individuals were filing reports with the national reporting center Stop AAPI Hate, which responds to and addresses hate incidents through research, advocacy, and capacity-building. During this time, The American Asian Foundation (TAAF) was founded. This organization became the driving force behind the team that decided to leverage X to provide a new window into how these incidents are personally affecting the people targeted.

The small team of two engineers looked to X as their source for incident data. They chose this platform for two reasons: firstly, it contains rich and diverse information, such as news, people's ideas, thoughts, daily life events, and things happening in their communities. Because of this, X offers a unique, intimate sample of various populations. X is also the only platform with a truly open API where the TAAF developers could access data and has in-depth tools to collect and analyze that data. 

Using the X API v2 search Tweets and Tweets lookup endpoints, the team was able to search post statuses and identify when people reported a hate incident (and even what hashtags they would use). This helped the team design a 1023-character search query that narrowed the billions of posts down to an amount the team could work with. 

Next came the task of verifying that the posts were indeed about hate incidents. The team developed a tool with natural language processing to help sift through thousands of posts to find ones relevant to the project. After applying their trained machine learning model, the posts were then verified by a human to further increase accuracy. 

In 2021, TAAF also co-authored the Documenting Anti-AAPI Hate Codebook with the Stop AAPI Hate coalition. This resource includes draft standards and practices for community-based data collection which helped the team classify incidents that people were sharing over X. 

From there, the next challenge was to present this data in a way that people could easily digest. TAAF created a data visualization tool, Decoding Hate, that displays their vast amount of data in an interactive way. With the help of a data visualization studio, they were able to take the annotated X data and turn thousands of posts into insightful stories that expose the truth behind what was happening within AAPI communities.