One of the key features of n8n is the ability to use Trigger nodes. Trigger nodes are special nodes that listen for events and then start a workflow. For example, you can build a workflow using the ClickUp Trigger node and a Google Sheets node. Whenever a new task gets created in ClickUp, the workflow would execute, and the data gets added to the Google Sheet. This makes Trigger nodes extremely useful, as you can monitor events across different services using them.
Once you start using several of these different Trigger nodes, you wonder what happens when you want to listen for an event from a service that does not have a Trigger node. For example, Google Sheets and Twitter (at the time of writing this tutorial - n8n@0.101.0) don't have a Trigger node.
In this article, we are going to discuss how you can monitor changes and execute workflows for services that do not support webhooks and don’t have Trigger nodes in n8n.
Introduction to polling
In a conventional n8n Trigger node, the node listens for events using a webhook. In this method, the service sends an update to the Trigger node when an event occurs. While polling works by periodically checking for new data by connecting to the server.
In this process, n8n will connect to the server every X time (every second/minute, etc) and check if there is any new data.
Not every platform supports sending updates via a webhook and this is where polling comes to the rescue. Using polling we can check for updates and execute our workflow if data gets added or updated.
How to poll using n8n?
If you break down the polling process, as described in the image above, you realize that it is possible to replicate the same functionality in n8n. The Interval node or the Cron node triggers the workflow periodically to fetch data from a service. We pass this data to the Function node, where all the magic happens.
In n8n, the getWorkflowStaticData() method allows access to the static workflow data. We can save the data directly with the workflow, but this data has to be small. On every execution, the Function node compares the incoming data with the data from the previous execution. If the data got changed, we pass it to the next node in the workflow. We also update the static data with this new data so that the next execution knows what data gets stored in the previous node. If the data did not get changed, you may return a message based on our use-case.
Trigger a workflow when new data gets added to your Google Sheets
I wanted to run a workflow every time there was new data added in Google Sheets, so I created a polling workflow. Follow along the steps mentioned below to learn to create a polling workflow.
Prerequisites
Set up n8n
Follow the instructions mentioned in the documentation to install and spin-up an n8n instance on your machine. You can also sign-up for a free n8n cloud trial to get access to our hosted service.
Set up a Google Sheet
Create a Google Sheet like this Google Sheet. We will monitor this sheet for new data that gets added. Feel free to make a copy of my Sheet.
We will also need to configure credentials for the Google Sheets node. For this tutorial, we will use the OAuth authentication method. You can learn to configure the OAuth credentials by following the steps mentioned in the documentation.
Set up Mattermost
We will send a message to a channel in a Mattermost team if new data gets added. Create a Mattermost team if you don’t have one already.
You will also have to configure the credentials for the Mattermost node. Follow the instructions mentioned in the documentation to configure the credentials.
Quick Start
If you don’t want to get into the details and have experience building workflows in n8n, you can follow the quick-start instructions below. If something is unclear or you want to learn more about how it works, feel free to dig deeper into the sections that follow.
Access your n8n instance, and copy and paste the workflow from the workflow page. Configure the following nodes:
- Google Sheets node
- Use the credentials that you configured earlier.
- Enter the Sheet ID of your Sheet in the Sheet ID field. If you’re not sure how to find the Sheet ID, refer to the section Google Sheets section below.
Note: If you change a column name in your Google Sheet, make sure to configure the Function node accordingly.
2. Mattermost node
- Use the credentials that you configured earlier.
- If you’re not the system administrator, enter the channel ID where you want the app to send the message in the Channel ID field. Otherwise, select the channel from the Channel ID dropdown list. Refer to the Mattermost section below, to learn more about the steps.
Save and execute the workflow!
Note: Activate the workflow to run it in production.
The Workflow
This workflow can be divided into three stages, as it progresses from start to finish:
- Read Data
- Extract New Data
- Communication
Stage 1 - Read Data
Interval node (Execute every 45 mins)
The Interval node triggers the workflow at a regular interval of time. For this tutorial, we want to execute the workflow every 45 minutes.
Click on the ‘+’ button on the top right corner and click on the Trigger tab. Select the Interval node from the list to add the node.
Select ‘Minutes’ from the Unit dropdown list and set the value of Interval to 45.
Rename the node’s headline from “Interval” to “Execute every 45 mins” by clicking the name, editing it, and clicking the ✔ (checkmark) to the right of the name. This will finish the configuration of the node and close the parameters window. Select ‘Save As’ from the Workflow menu on the left sidebar. Once you save the workflow, click on the Execute Node button to execute the node.
Google Sheets node (Read data)
The Google Sheets node in n8n allows you to create new sheets and read, lookup, append, delete, and update data from a sheet. Our workflow will read the data from a Google Sheet every 45 minutes.
To add the Google Sheets node, click on the ‘+’ button, and select the Google Sheets node from the Regular section. Use the credentials that you configured earlier.
Copy the string of characters located between /d/ and /edit in your spreadsheet URL. This string is the Sheet ID. For example, for the URL https://docs.google.com/spreadsheets/d/1PyCgaglXSCbxVmHuwFbkKDF9e3PW_iUn8T-iAd_MYjQ/edit#gid=0 the Sheet ID is 1PyCgaglXSCbxVmHuwFbkKDF9e3PW_iUn8T-iAd_MYjQ. Enter this Sheet ID in the Sheet ID field in the Google Sheet node. Enter the range of the columns in the Range field. The range contains the column references corresponding to those on the spreadsheet. The range tells the Google Sheets node from which columns to read the data.
Rename the node to Read data and click on the Execute Node button. The node will return the data from the Google Sheet.
Stage 2- Extract new data
Function node (Check if new data)
Our workflow reads the data from a Google Sheet and returns all the data. However, we only want the data that was recently added. We will use the Function node to return only the data that was added after the previous execution.
To help you understand this better, let us take a look at an example. Our workflow ran at 1:00 PM and returned the following data:
[
{
"id": 0,
"name": "Sam",
"email": "sam@testmail.com"
},
{
"id": 1,
"name": "Ram",
"email": "ram@testmail.com"
}
]New data was added to our Google Sheet at 1:20 PM. When your workflow will execute next, we only want this new data. Using the Function node we will return this new data. n8n has a getWorkflowStaticData() method that gives access to the static workflow data. We can save data directly with the workflow, however, this data should be very small. When the workflow execution succeeds, n8n will check automatically if the data has changed and will save it, if necessary.
Click on the ‘+’ icon and select the Function node from the list. In the Javascript Code editor, paste the following code snippet.
const new_items = [];
// Get static data stored with the workflow
const data = $getWorkflowStaticData("node");
data.ids = data.ids || [];
for (let i = items.length - 1; i >= 0; i--) {
// Check if data is already present
if (data.ids.includes(items[i].json.ID)) {
break;
} else {
// if new data then add it to an array
new_items.push({
json: {
id: items[i].json.ID,
name: items[i].json.Name,
email: items[i].json.Email
},
});
}
}
data.ids = items.map((item) => item.json.ID);
// return new items
return new_items;The above code snippet gets the static workflow data and checks for new items. If new items are found, it is added to an array and gets returned by the node. If the data remains unchanged, an empty array is returned by the node.
Rename the node to Check if new data and click on the Execute Node button.
Note: The static data can not be read and written when we’re building the workflow. Hence, on every execution, the node will return all the data. When the workflow is set to active and triggered by a Trigger node, the node will save the static data and return only the new data that was added (if any).
The following image shows the output returned by the node when the workflow is not active.
The following image shows the output returned by the node when the workflow is active.
Stage 3- Communication
Mattermost node (Send message)
If new data was added to the Google Sheet we will send a message to Mattermost along with the new data. We will send a message to the updates channel. But you can select a different channel to send the message.
Click on the ‘+’ button and select the Mattermost node. Connect the node to the true branch of the Is new item? node. Use the credentials you configured earlier.
If you are not the system administrator of the Mattermost team, navigate to Mattermost and select the ‘updates’ channel from the left sidebar. Click on the caret next to ‘updates’ and select ‘View Info’ from the dropdown list. Copy and paste the displayed ID in the Channel ID field in the Mattermost node.
If you’re the system administrator, select the ‘updates’ channel from the Channel ID dropdown list.
Click on the gears icon next to the Message field and select ‘Add Expression’. Enter the following expression in the Expression Editor.
New information was added to your Google Sheet.
ID: {{$json["id"]}}
Name: {{$json["name"]}}
Email: {{$json["email"]}}Rename the node to Send message and click on Execute Node to run the node.
What’s next?
Currently, the Check if new data node in our workflow returns all the items since the static data is not getting saved. Save the workflow and toggle the Active, on the top right, to true. This will set our workflow to active and our workflow will execute every 45 mins. Add new data to the Google Sheets, and after 45 mins (for testing you can set the time to 2 mins!) you will receive a message on Mattermost.
Such polling workflows can be used for various use-cases. If you’re using Google Forms to collect data, and if the form adds the data to a Google Sheet, you can build a workflow that will trigger the workflow whenever there is a new form response.
Another interesting use-case can be for Twitter. You can build a polling workflow that executes every 10 mins. This workflow would fetch Tweets for your search criteria using the Twitter node and return the most recent tweets. At n8n, we use a similar workflow that sends a message to Mattermost with the recent Tweets. You can find the workflow on the workflow page.
A few more examples where polling can help:
- Trigger a workflow when an event gets added to your Google Calendar using the Google Calendar node
- Trigger a workflow when price changes for a cryptocurrency using the CoinGecko node
- Trigger a workflow when new data gets added to your database (MongoDB, Postgres, etc.)
Summary
In this tutorial, we learned how polling can help us overcome a barrier for the nodes that do not have a Trigger node. We can use polling with the nodes that do not have a default Trigger node and build a workflow based to trigger actions when new data is added or updated.
We built a polling workflow that executes every 45 mins and reads data from a Google Sheet. If the data has been updated or new data has been added, the workflow sends a message to Mattermost.
I’d like to know about how you are using the concept of polling and building workflows with n8n! If you run into any issues while following the tutorial, feel free to message me on Twitter or ask for help on our forum 🧡
Start automating!
The best part is, you can start automating for free with n8n. The easiest way to get started is to sign up for a free n8n cloud trial. Thanks to n8n’s fair-code license, you can also self-host n8n for free.