No code sentiment analysis
This tutorial shows how to build a data processing pipeline without code. You’ll analyze tweets that contain information about Bitcoin and stream both raw and transformed data into Snowflake, a storage platform, using the Twitter, HuggingFace and Snowflake connectors.
I’ve made a video of this tutorial if you prefer watching to reading.
What you need for this tutorial
-
Twitter developer account (You can follow this tutorial to set up a developer account)
Step one: create a database
Sign in to your Snowflake account to create the Snowflake database which will receive your data. Call this "demodata" and click "Create."
Step two: get your data
In the Quix Portal, navigate to the Code Samples
and search for the Twitter source
connector.
Click "Add new." This adds the source to your pipeline and brings you back to the Code Samples.
Fill in the necessary fields:
-
Name: Twitter Data - Source
-
Output: twitter-data
-
Twitter bearer token: paste your Twitter Dev token here
-
Twitter_search_paramaters: (#BTC OR btc #btc OR BTC)
Tip
Use search parameters to obtain Tweets on a subject that interests you! e.g. (#dolphins OR dolphins)
Click "Deploy"
Step three: transformation for sentiment analysis
-
Click the "Add transformation" button
-
In the Code Samples, search for "HuggingFace"
-
Click "Set up and deploy" on the HuggingFace connector
-
Choose "Twitter data" as the input topic
-
The output field should be set to "hugging-face-output" by default, leave this as it is or enter this if it’s not pre-populated.
-
Leave all other values with their defaults
-
Click "Deploy"
Note
Find out more about the hugging face model and the other models you could use at huggingface.co
Step five: delivery to your Snowflake database
-
Click the "Add destination" button on the home screen
-
Search the Code Samples for the Snowflake connector
-
Click "Set up and deploy" on the connector
-
Fill in the necessary fields:
-
Choose the hugging-face-output output topic
-
The "Broker__TopicName" field should be set to "hugging-face-output". This means it will receive the data being output by the sentiment analysis model.
To fill in the Snowflake locator and region (these are similar to a unique ID for your Snowflake instance), navigate to your Snowflake account. Copy the locator and region from the URL and paste them into the corresponding fields in the connector setup in Quix. Lastly, input your username and password.
Click "Deploy" on the Snowflake connector. If the credentials and connection details are correct, you’ll see the "Connected" status in the log and will be redirected to your workspace.
Congratulations! You built a no-code pipeline that filters and collects data from Twitter, transforms it with a HuggingFace model and delivers it to a Snowflake database.
You can now go back over to Snowflake and find the "Databases" menu. Expand the "demodata" database and then find the tables under "public".
Tip
If you need any help, please sign up to the Quix community forum.