Process - threshold detection
In this part of the tour you'll learn how to create a transform. The transform detects if CPU load exceeds a certain threshold, and if so, sends a dataframe to its output topic.
Watch the video
Create the transform
To create the threshold detection transform:
- In your
Develop
environment, click onCode Samples
in the main left-hand navigation. - Select the
Python
,Transformation
, andBasic templates
filters. - For
Starter transformation
clickPreview code
. - Click
Edit code
. - Name the transform "CPU Threshold".
- Select the input topic
cpu-load
. - For the output topic, add a new topic called
cpu-spike
. - Click
Save as Application
. - In the project view click on
main.py
to edit it. -
Replace all the code in
main.py
with the following:import quixstreams as qx import os import pandas as pd client = qx.QuixStreamingClient() topic_consumer = client.get_topic_consumer(os.environ["input"], consumer_group = "empty-transformation") topic_producer = client.get_topic_producer(os.environ["output"]) def on_dataframe_received_handler(stream_consumer: qx.StreamConsumer, df: pd.DataFrame): cpu_load = df['CPU_Load'][0] print(f"CPU Load: {cpu_load} %") stream_producer = topic_producer.get_or_create_stream(stream_id = stream_consumer.stream_id) if cpu_load > 50: # hard-coded threshold print(f"CPU spike of {cpu_load} detected!") stream_producer.timeseries.buffer.publish(df) def on_stream_received_handler(stream_consumer: qx.StreamConsumer): stream_consumer.timeseries.on_dataframe_received = on_dataframe_received_handler topic_consumer.on_stream_received = on_stream_received_handler print("Listening to streams. Press CTRL-C to exit.") qx.App.run()
-
Tag the project as
process-v1
and deploy as a service (watch the video if you're not sure how to do this). - Monitor the logs for the deployed process.
Generate a CPU spike
You can generate a CPU spike by starting up several large applications. In the logs you will see a message similar to the following when a spike is detected:
🏃♀️ Next step
Create a destination to log events and send a notification SMS!