# Forloop — full corpus > Self-healing web scraping data feeds. Forloop is web scraping data infrastructure. Teams build scrapers, APIs and data workflows once. Forloop maintains those scrapers when websites and APIs change, checks the data, and keeps the feeds running. Topics: web scraping, scraper maintenance, data infrastructure. Index: https://www.forloop.ai/llms.txt ## Web scraping Forloop scrapes websites, files and APIs into structured data. Scrapers are built visually or in Python, then deployed as jobs and APIs. Primary pages: https://www.forloop.ai/docs/getting_started, https://www.forloop.ai/docs/webscraping_rpa, https://www.forloop.ai/docs/browser_view, https://www.forloop.ai/docs/data_sources, https://www.forloop.ai/developers ## Scraper maintenance When a page or API changes, Forloop repairs the connector during the run instead of leaving a broken scraper. A data-quality check runs after the repair. Broken connectors are counted from the live maintenance counter on the homepage. Primary pages: https://www.forloop.ai, https://www.forloop.ai/developers, https://www.forloop.ai/docs/control, https://www.forloop.ai/docs/triggers, https://www.forloop.ai/docs/monitoring ## Data infrastructure Scrapers sit inside Python-first data infrastructure: pipelines, orchestration, databases, integrations and APIs. Products such as Digishelf, real estate feeds and logistics feeds are applications on that infrastructure. Primary pages: https://www.forloop.ai/docs/pipelines_content_table, https://www.forloop.ai/docs/integrations, https://www.forloop.ai/docs/databases_content_table, https://www.forloop.ai/solutions/digishelf, https://www.forloop.ai/real-estate, https://www.forloop.ai/logistics ## Documentation --- URL: https://www.forloop.ai/docs Title: Welcome to Forloop.ai’s documentation! Section: Documentation --- # Welcome to Forloop.ai’s documentation! There is no Frigate like a Book To take us Lands away, Nor any Coursers like a Page Of prancing Poetry – This Traverse may the poorest take Without oppress of Toll – How frugal is the Chariot That bears a Human soul. —Emily Dickinson, There is no Frigate like a Book First steps: - What is Forloop? - Getting started Node Catalogue (Build): - Webscraping and RPA - Data sources - Cleaning - Control - Integrations - Custom Content tables: - Pipelines - Scripts (Beta) - Datasets (Beta) - Databases (Beta) - Machines (Beta) View modes (View): - Code View - Data Grid View - Database View - Browser View Databases: - User Guide: Adding a Database Connection - Supported Database Dialects - Working with Databases Guide Analyze: - Wizard - Monitoring Orchestration: - Triggers Help: - Forloop.ai main page - Contact us --- URL: https://www.forloop.ai/docs/forloop Title: What is Forloop? Section: First steps --- # What is Forloop? Welcome to the world of Forloop, a powerful no-code platform designed to revolutionize your data collection, preparation, and automation processes. In this introductory chapter, we will delve into the essence of Forloop and how it can empower you to leverage external data sources, streamline your workflows, and seamlessly integrate with popular data storage platforms such as databases, Google Sheets, Airtable, and more. ## Introducing Forloop Forloop is a cutting-edge no-code tool that empowers individuals and businesses to harness the full potential of data-driven decision making. Whether you are a data analyst, a business owner, or simply someone who seeks to streamline their data processes, Forloop offers a comprehensive suite of tools and functionalities to make your journey seamless and efficient. ## The Power of External Data One of the key strengths of Forloop lies in its ability to tap into external data sources. By connecting to various data streams, APIs, and web scraping techniques, Forloop enables you to gather real-time, relevant data from a wide range of sources. This means you can access up-to-date information, monitor market trends, and stay ahead of the competition effortlessly. ## Streamlining Data Preparation Collecting data is only the first step. Forloop understands the importance of data preparation and provides a robust set of features to help you clean, transform, and structure your data effortlessly. With its intuitive interface and powerful data manipulation capabilities, Forloop allows you to filter, aggregate, and combine datasets to derive meaningful insights, without the need for complex coding. ## Automation Made Easy Manual, repetitive tasks can be time-consuming and prone to errors. Forloop eliminates these challenges by offering seamless automation capabilities. From data extraction to analysis and reporting, Forloop enables you to automate your workflows, saving you valuable time and ensuring accuracy in every step. With just a few clicks, you can schedule recurring tasks, set up triggers, and let Forloop handle the rest. ## Integrations with External Data Storages Forloop understands the importance of centralized data management. That’s why it seamlessly integrates with popular data storage platforms such as databases, Google Sheets, Airtable, and more. With these integrations, you can easily import and export data, maintain data integrity, and collaborate with your team in real-time. ## Embrace the Power of Forloop As you embark on your Forloop journey, you will discover endless possibilities for data-driven automation. Whether you are a seasoned data professional or a novice looking to harness the power of data, Forloop equips you with the tools and capabilities to transform the way you work with data. In the upcoming chapters, we will explore the features, functionalities, and best practices of Forloop, providing you with a comprehensive guide to unleash the full potential of this remarkable platform. So, let’s dive in and explore the world of Forloop, where data-driven automation becomes a reality. Tip Don’t know where to continue? Try the Getting started section. It will guide you through the realms of the platform so you will become a pro in no time. --- URL: https://www.forloop.ai/docs/getting_started Title: Getting started Section: First steps --- # Getting started We understand how daunting it can be to open a completely new program and try to make sense of it. It’s akin to venturing into an exotic country with a different culture, language, and origins. However, there’s no need to worry. You’re not alone in this endeavor. We’re here to assist you every step of the way, ensuring that your first experience with our platform is smooth and comfortable. Consider us your trusted guide, helping you feel at home in no time. So go ahead, grab a cup of ☕, and let’s get started! ## Do not go gentle into that good night… When you go to the Forloop platform’s start page you will have three options how to continue: - Getting Started - this option will take you to a step-by-step tutorial and leave you with a basic knowledge about the platform elements and a simple pipeline ready - you can find a step-by-step guide through this tutorial in the chapter Tutorial walkthrough - No Code - this option takes you straight to the cloud version of Forloop - if you want to start here, the Meet Forloop Cloud section might come in handy - Desktop - this options will take you to a download page of the desktop version of Forloop - the Meet Forloop Desktop section might come in handy in this case Tip If you are completely new to the platform we recommend choosing the first option and see Tutorial walkthrough. ## Tutorial walkthrough In this guide, we will walk you through the process of selecting elements from a webpage(s) of your choice and extracting data from it (them). Moreover, you will learn how to save the extracted data as either a CSV file or directly to Google Sheets. The final step of the tutorial will introduce you to the platform itself, where you’ll find a freshly generated no-code pipeline encompassing the entire process. In a nutshell, by following this tutorial, you will: - Learn how to easily select webpage elements using the Forloop browser view - Perform your first data extraction with Forloop, marking an exciting milestone 🎉 - Preview the extracted data in an elegant tabular structure - Save the data locally or send it to Google Sheets Not bad for just a few minutes of work, right? Note Currently, we have one tutorial available that focuses on web scraping use cases. However, in the near future, we have plans to release additional tutorials catering to various use cases. So, stay tuned for upcoming tutorials! ### Step 1 - Enter a website At the first step you need to enter URL of the website you want to scrape. We chose https://www.forloop.ai/blog but you can choose whichever website you like. After entering the website in the field, click continue. ### Step 2 - Pick elements After loading the website, you will be prompted to select the elements that Forloop should search for. You have the option to select all possibilities, but you can also choose specific elements if you have a clear idea of what you are looking for. When all of the desired elements are selected, continue with confirm. ### Step 3 - Select website’s elements for scraping After a short while a preview of the webpage will be loaded, in our case of https://www.forloop.ai/blog: You can actively select the elements which you would like to scrape as shown here: If you want to select all elements of a certain type (for example, the titles of all articles listed on the blog), there’s no need to manually select each one individually. Instead, you can select just one element of that type, then click on FIND SIMILAR ITEMS, and let Forloop take care of the rest. All similar items selected by Forloop will be marked by yellow rectangles. ### Step 4 - Pagination mode At this stage, you can choose whether you want to scrape only the single page of the website you entered or multiple pages. To include more pages, you have the option to enter the URLs manually or import them from a CSV file. In our example, we focused on scraping a single page, but feel free to explore other options and experiment with different settings. ### Step 5 - Choose destination Now that you have the data, the next step is to determine where you would like to store it. Currently, two options are available: - Save the data locally to a CSV file. - Store the data in a Google Sheet. In our example, we chose the Google Sheet option. To proceed with this option, you will need an existing Google Sheet that is set to be shared with anyone who has the link. Accessing sharing options of a Google sheet Setting sheet’s sharing option to Anyone with the link After setting up the Google sheet all you have to do is pasting it’s URL and the data are on their way! 🚀 Insert previewed data to Google sheet and continue to the final step. ### Step 6 - Finalize pipeline Congratulations! You have successfully completed your first quest ⚔️🧙‍♂️. Now, it’s time to claim your well-deserved reward! Your reward is a complete pipeline of the entire configuration process you just completed, awaiting you in the platform. So, without further ado, click that continue button, collect your reward, and embark on another mighty quest by exploring the platform! 🐉 ## Get to Know Forloop Okay, so you’ve got your copy of Forloop installed or a browser with the cloud version opened, you are registered, loged-in, maybe energized by succesfully finishing the tutorial and… now what? Forloop greets you with a nice clean working grid accompanied by all kinds of buttons doing all kinds of useful things. But how to get a grip of all of this? Let’s take a little tour! Tip If you are in a cloud version, continue with the following chapter. For the desktop version guide, see Meet Forloop Desktop. ### Meet Forloop Cloud ☁️ Forloop Cloud can be considered as the younger sibling of Forloop Desktop. While it may not be as mature and powerful as its counterpart, it is highly motivated and continuously improving. If you’re not a power user who enjoys delving into every feature of the tool but appreciates a beautiful design and user-friendly interface, then you’re in the right place. Meet Forloop Cloud ☁️. #### Menus and Nodes Within the platform, you’ll find several menus, each containing numerous nodes. The organization of nodes within each menu is not random; rather, they are grouped based on their scope of usage. For example, you’ll find webscraping and RPA nodes located under the “Webscraping & RPA” menu. We encourage you to click through the menus and explore the functionalities they offer. This will allow you to familiarize yourself with the range of available nodes and their respective purposes. Take your time to navigate through the menus and discover the capabilities of each. Node menus in Forloop Cloud ☁️ Hint For some general usage (how to create a node, how to connect more nodes into a pipeline etc.) see Build Your First Pipeline. #### View Modes Despite it being pretty useful, sometimes the workspace grid is not enough. In some occasions we would like to view loaded data in a data table, view loaded databases and their tables, code of the loaded nodes or scraping functionalities in a built-in browser. Luckily, all of these cases are covered by View buttons! View modes in Forloop Cloud ☁️ In addition, not only do they add some additional viewing options but they offer additional functionalities as well! See: - Code View chapter to find out how Forloop builds a bridge between code and no-code - Browser View chapter to find out how you can interactively select elements of the webpage in Forloop and scrape their data contents - Grid View chapter to find out how you can view and work with data in an elegant table format - Database View chapter to find out how you can show and work with databases of various dialects Tip For some general usage of View options see Working with View Options. ### Meet Forloop Desktop 🖥️ In the era of cloud services, the convenience of accessing applications and data online is undeniable. However, there are situations where having a locally installed software on your machine can be advantageous. If you are a power user who likes delving into every feature the platform offers or you seek stability and safety offered by a desktop app, you are at the right place. Meet Forloop Desktop 🖥️ #### Installation (desktop version) Supported platforms: Windows, MacOS. Donwnload stable release of Forloop from: http://app.forloop.ai/ - Run the downloaded file to install the platform - If on Windows - do not install into Program Files folder! - Launch the platform - Register on http://app.forloop.ai/register (if not done before) - Use registration details to log in Forloop.ai platform - Enjoy Forloop! 🚀 Important In case of encountering any issues during installation, please contact us. #### Menus and Nodes Within the platform, you’ll find several menus, each containing numerous nodes. The organization of nodes within each menu is not random; rather, they are grouped based on their scope of usage. For example, you’ll find webscraping and RPA nodes located under the “Webscraping & RPA” menu. We encourage you to click through the menus and explore the functionalities they offer. This will allow you to familiarize yourself with the range of available nodes and their respective purposes. Take your time to navigate through the menus and discover the capabilities of eac Node menus in Forloop Desktop 🖥️ Tip For some general usage (how to create a node, how to connect more nodes into a pipeline etc.) see Build Your First Pipeline. #### View Modes Despite it being pretty useful, sometimes the workspace grid is not enough. In some occasions we would like to view loaded data in a data table, view loaded databases and their tables, code of the loaded nodes or scraping functionalities in a built-in browser. Luckily, all of these cases are covered by View buttons! View modes in Forloop Desktop 🖥️ In addition, not only do they add some additional viewing options but they offer additional functionalities as well! See: - Code View chapter to find out how Forloop builds a bridge between code and no-code - Browser View chapter to find out how you can interactively select elements of the webpage in Forloop and scrape their data contents - Grid View chapter to find out how you can view and work with data in an elegant table format - Database View chapter to find out how you can show and work with databases of various dialects Tip For some general usage of View options see Working with View Options. #### Variable Explorer For storing and working with data Forloop presents the Variable Explorer. It stores data of various types (numbers, strings, lists etc.) in a form of color coded rectangles. These can be the interactively used in Forloop nodes. Let’s take a look at how the variables can be added. It can serve as a good training of creating a node as well. So, let’s find a Create New Variable node first. Clicking on an item in one of the menus creates a corresponding node. By clicking on the node in the working space we select it (its edges get colored). When a node is selected its UX appears above the variable explorer. In our case, a New Variable node, we just need to enter a name of the new variable and its value. Its type will be detected automatically. The whole process is shown in a short GIF below. Note New Variable node is not the only way of adding new variables to variable explorer! There are in fact many other options like Load Excel/CSV, Define function, New Datetime etc., which all create a new variable (dataframe, function, datetime, …). See Build Your First Pipeline or documentation of specific noodes for more information. ## Build Your First Pipeline In previous sections we here and there mentioned Forloops capability of building pipelines. After all it’s mentioned in the What is Forloop? section that it is one of the main features. So how does it work? In previous sections have we seen a very basic creation of node and an interaction with it. But since working manually with only one single node at a time would be extremely dull, Forloop introduces an extremely elegant and easy way to build pipelines from these nodes. The process is fairly simple. Let’s start by explaining how the nodes get connected. ### Connecting the Nodes In order to connect two nodes with the idea being that the Node 1 will run first and after its task is done then the Node 2 will do its duty. To setup this process 3 elementary steps are required: - create two or more nodes as shown in previous sections - with a double-click on one of the nodes you bring it to the “ready to connect” state (its edges is colored differently than in case of a single click) - after we click on another node after a double-click on the first one both nodes get connected (we see it as an arrow) Note Building even an infinitely long pipeline can be done just by iteratively reapeating this process. However the Forloop team strongly discourage you from building an infinitely long pipeline as it can turn out to be a little time consuming. ### Running the Pipeline Okay, now we’ve got some nodes connected but what now? Now we can use the Pipeline Control Options in the upper panel. We can either save the pipeline if we like it, empty it if we don’t, load a new saved one or run the one present in our working space. The last option is what we will use. ### Building a Basic Notepad Opening Pipeline By combining all the knowledge we collected so far building a simple pipeline should be a piece of 🍰. Let’s build a really elementary one which only opens a notepad (or its macOS/Linux counterpart). What we need is: - Use key node to hit win button (or cmd + space combination to open Spotlight on macOS) - Write node to write notepad (or some other app) into search - Use key again to hit enter (return) ## Working with View Options --- URL: https://www.forloop.ai/docs/webscraping_rpa Title: Webscraping and RPA Section: Node Catalogue --- # Webscraping and RPA ## Click ### Description Click Node does a mouse (double)click on a given coordinates. ### Parameters Two parameters, a pair of coordinates count in pixels from upper left corner, are required: Parameter Type Description X (required) integer A horizontal coordinate count in pixels from left edge of the display. Y (required) integer A vertical coordinate count in pixels from the upper edge of the display. Double click bool (checkbox) If checked (True): mouse will perform a double-click, else single-click ## Click Image ### Description Finds provided image on currently scraped page and clicks at point that has coordinates (image_pos_X + X, image_pos_Y + Y). In other words icon finds the coordinates of image on page and clicks at point (X, Y), relatively to found image. ### Parameters Parameter Type Description Image path (required) string Path to image Click offset X (required) int/float Relative to image offset on axis X Click offset Y (required) int/float Relative to image offset on axis Y ## Wait ### Description Wait Node creates a pauses between pipeline steps. It waits (and does nothing, obviously) for a specified amount of ms and then lets the next step in a pipeline to proceed. ### Parameters Two parameters are required: Parameter Type Description Miliseconds (required) int/float Waiting time interval in miliseconds, e.g. entry: 1000 → 1000 ms (1 s) waiting time before another step. Add random ms int/float Adds a random real picked from a uniform distribution defined on the interval (- entered value, + entered value). ## Write ### Description Types the text given via the entry box as if it were typed on the regular keyboard. ### Parameters Parameter Type Description Text (required) string The text to be typed by the write icon ## Use Key ### Description Hits a single key or a combination of keys on a keyboard. At least one key must be entered and a combination (series) of up to three keys can be set up. ### Parameters Parameter Type Description Key 1 (required) comboentry value The keyboard key to be clicked or, if other keys entered, the first key of a combination. Can be either selected from the combobox or written in manually. Key 2 comboentry value The second key of the combination. Can be either selected from the combobox or written in manually. Key 2 comboentry value The third key of the combination. Can be either selected from the combobox or written in manually. ## Open Browser ### Description Opens the system default web browser (for further usage such as Load Website etc.). Allows to choose driver to use. For now available: Firefox (Geckodriver), Chrome. Note: To use a driver an appropriate web browser should be installed. Driver itself will be installed automatically. ### Parameters Parameter Type Description Show browser boolean Show browser GUI Driver driver (combobox) Driver type (Firefox, Chrome) ## Load Website ### Description Loads website with a given URL. ### Parameters One parameter is required: Parameter Type Description URL (required) string URL of the loaded website, e.g. https://forloop.ai/. Take screenshot boolean If screenshot is needed (e.g. for using in Browser View) ## Refresh Page Source ### Description Refreshes source code of the loaded website. ## Click XPath Element ### Description Clicks on the website element specified by its XPath. ### Parameters Parameter Type Description XPath (required) string The XPath of the element to be clicked upon, e.g. //div[ class=”media-select__input-content”]//button. ## Scan Web Page ### Description Searches for different elements on page and highlights them in Browser View. Available types of elements: - Tables - Bullet lists - Texts - Headlines - Links - Images - + elements with custom XPath ### Parameters Parameter Type Description Tables boolean Include tables in search Bullet lists boolean Include bullet lists in search Texts boolean Include texts in search Headlines boolean Include headlines lists in search Links boolean Include links in search Images boolean Include images in search Custom XPath (optional) string Include elements with custom XPath ## Extract XPath Element ### Description Finds HTML element using XPath and exports it. Also allows to load XPath of element from Browser View to the first parameter (XPath). ### Parameters Parameter Type Description XPath (required) string XPath of element to be extracted Output variable (required) string Name of variable to be created inside platform Output filename (required) string Name of file to be created Write in file mode string (combobox) File mode to use: - w+ (rewrite previous file content), - a+ (append to previous file content) ## Extract Multiple XPaths ### Description Finds multiple HTML elements using XPath and exports them. ### Parameters Parameter Type Description Extraction setup file path (required) string Path to .txt file with XPaths (1 per row) Output variable (required) string Name of variable to be created inside platform ## Extract Table XPath Element ### Description Finds table using XPath and exports it. Also allows to load XPath of element from Browser View to the first parameter (XPath). ### Parameters Parameter Type Description XPath Rows (required) string XPath of table row XPath Columns (required) string XPath of table column Output variable (required) string Name of variable to be created inside platform ## Set Proxy ### Description Set’s a new proxy connection. ### Parameters Parameter Type Description IP (required) string IP of the proxy server. Port (required) integer Proxy server port. Username string Login username. Password string Login password. ## Click Name Element ### Description Clicks on the website element containing the given text. ### Parameters Parameter Type Description Text (required) string The text to be searched on the website and clicked upon. ## Click Id Element ### Description Clicks on the website element specified by its ID. ### Parameters Parameter Type Description Id string The ID of the element to be clicked upon. ## Get Current URL ### Description Saves the URL of web page, which is being currently scraped, to new variable inside platform. ### Parameters Parameter Type Description Output variable (required) string Name of variable to be created inside platform ## Close Browser ### Description Closes browser after all scraping processes are done. ## Wait Until Element Is Located ### Description Creates a pause until a certain element is located. Can be useful for instance when dealing with popup windows on page. ### Parameters Parameter Type Description XPath (required) string XPath of element that should appear on page ## Extract Page Source ### Description Extracts the HTML source of currently scraped web page and saves it to new variable inside platform. ### Parameters Parameter Type Description Output variable (required) string Name of variable to be created inside platform ## Scroll Web Page ### Description Scrolls web page by X pixels in directions Up/Down. ### Parameters Parameter Type Description Scroll to string (combobox) The direction of scrolling (Up/Down) Scroll by (required) int Length of scrolling in pixels Scroll max boolean Scroll to the top / end of page ## Create Folder ### Description Creates a new folder with a given name at a given location. ### Parameters Two parameters, a name of the new folder and its location are required. Parameter Type Description Folder location (req.) string Path to a directory in which the new folder will be created, eg. /Users/MrForloop/new_dir_place. Folder name (req.) string A name of the new folder, eg. my_new_dir. The final path to the new directory will be in the example case /Users/MrForloop/new_dir_place/my_new_dir. ## Move File ### Description Moves a file from it’s current directory to a different one. ### Parameters Parameter Type Description File (req.) string Path to a file to be moved, eg. /Users/awesome_file.flpl. Move to (req.) string Path to a directory to which the file will be moved into, eg. /Users/MrForloop. The final path to the moved file will be in the example case /Users/MrForloop/awesome_file.flpl. ## Create File Queue ### Description Creates a file queue folder whit a specified template suffix, i.e. .txt. Files in this queue can be further processed directly in Forloop. ### Parameters Parameter Type Description Root Folder (req.) string Path to a root folder where the queue directory will be created, eg. /Users/MrForloop/Desktop/. Queue name (req.) string A name of the to be created queue directory. Suffix combobox option Only files with this specific suffix in the queue folder will be processed via Process Item In Queue. ## Process Item In Queue ### Description Processes a next-in-the-line file in the file queue. ### Parameters Parameter Type Description New variable name (req.) string Name of the variable storing the data from the processed file. ## Delete File ### Description Deletes a specified file as if by using rm command on UNIX systems or del on Win/DOS). ### Parameters Parameter Type Description File name (req.) string Path to the file, i.e. /Users/Delete/This/file.txt. ## Download Image ### Description Downloads image from URL. ### Parameters Parameter Type Description Image URL (required) string URL with image Output filename (required) string Name of image to be saved ## Download Images XPath ### Description Downloads image using XPath. Can be used for downloading multiple images at once. ### Parameters Parameter Type Description XPath (required) string XPath of image element Output filename (required) string Name of image to be saved. In case of downloading multiple images will be used as a directory name. --- URL: https://www.forloop.ai/docs/data_sources Title: Data sources Section: Node Catalogue --- # Data sources ## Add Database ### Description Add Database connects a database to Forloop. Either a new database can be entered via the New Database button or an already existing database can be selected from a combobox. ## Add SQL Server DB ### Description Add SQL Server DB Icon connects a SQL Server database to Forloop. Either a new database can be entered via the New Database button or an already existing database can be selected from a combobox. ## Add MySQL DB ### Description Add Database Icon connects a MySQL database to Forloop. Either a new database can be entered via the New Database button or an already existing database can be selected from a combobox. ## Add Mongo DB ### Description Add Database Icon connects a Mongo database to Forloop. Either a new database can be entered via the New Database button or an already existing database can be selected from a combobox. ## Add Snowflake Warehouse ### Description Add Database Icon connects a Snowflake warehouse to Forloop. Either a new database can be entered via the New Database button or an already existing database can be selected from a combobox. ## Load Excel/CSV ### Description Load Excel/CSV Node serves to loading a .xlsx or .csv file for further cleaning and analysis. Hint For a detailed walkthrough see the step-by-step guide. ### Parameters Load Excel/CSV Node requires 1 parameter to succesfully load data from .xlsx or .csv file. Parameter Type Description File name (required) string A path to the data file must be filled in into the Filename entry either manually or via file explorer by clicking the Look up file button below the entry line. Load as string A name of the dataframe variable containing the data. ### Step-by-step guide ## Save Excel/CSV Parameter Type Description Value (required) list/dict/DataFrame A variable containing the data to be stored into a CSV/xlsx file. Filename string A path of the newly created file containing the data. If file doesn’t exist it will be created. ## Get API (REST) ### Description Get API icon makes a get request from a website defined by its URL and (optionally) the API’s header. The response of the get request in form of a json (dictionary) is saved as a new variable in the variable explorer for further usage. ### Parameters Get API node requires at least 2 parameters. The header parameter is optional. Parameter Type Description URL string A URL of the desired websites API. Header (optional) string A dictionary of HTTP headers to send into the request, e.g. {“content-type”:”text”}. Save as string A name of the new variable containing the response dictionary. ## Post API (REST) ### Description Post API icon makes a post request to a website defined by its URL. The response of the post request in form of a json (dictionary) is saved as a new variable in the variable explorer for further usage. ### Parameters Post API node requires 3 parameters: Parameter Type Description URL string A URL of the desired websites API. Data string A path to the .txt or .csv file containing the data for post request. Save as string A name of the new variable containing the response dictionary. ## Delete API (REST) ### Description Delete API icon posts a delete request to a website defined by its URL. ### Parameters Post API node requires at least 1 parameter. The header parameter is optional. Parameter Type Description URL string A URL of the desired websites API. Header (optional) string A dictionary of HTTP headers to send into the request, e.g. {“content-type”:”text”}. ## Put API (REST) ### Description Put API icon makes a put request to a website defined by its URL. ### Parameters Put API node requires 2 parameters: Parameter Type Description URL string A URL of the desired websites API. Data string A path to the .txt or .csv file containing the data for put request. ## Dataframe to List ### Description Takes a column from a dataframe and transforms it into a list which is then stored as a new variable in the variable explorer. ### Parameters Parameter Type Description Column name string A name of the column to be transformed into a list. New variable name string A name of the new variable containing the list (column in the dataframe). ## List to Dataframe ### Description Converts a variable of type list into a new DataFrame variable. ### Parameters Parameter Type Description List (required) list A list variable to be conveerted. New column names list of strings A list of strings that will be appointed as column names of the new DataFrame variable. Important: the length of this list should be equal to the length of the list variable! New variable name string A name of the new variable containing the DataFrame. ## Load Txt File ### Description Loads a TXT file and stores its contents to a string variable. ### Parameters Parameter Type Description File name (required) string A path leading to the TXT file. Variable name string A name of the new variable containing the contents of the TXT file. ## Save Txt File ### Description Takes a variable from Forloop, converts it to a string format and saves it into a TXT file. ### Parameters Parameter Type Description Value (required) string/list/dict/numeric A variable to be saved into a TXT file. Filename (required) string A path of the TXT file. If the file does not exist, it will be created. Write in File Modes combobox option a+ = file created if doesn’t exist and text appended at the end of the file, w+ = file created if it doesn’t exist but if it does, it get’s overwritten ## Load Json File ### Description Loads a contents of a JSON file into a dict or list variable. ### Parameters Parameter Type Description File name (required) string A path of the JSON file. Variable name string A name of the new variable storing the parsed contents of the JSON file, e.g. my_test_json_data. ## Save Json File ### Description Takes a dictionary variable as an input, parses it to JSON format a saves it to a JSON file. ### Parameters Parameter Type Description Dictionary (required) dict A dict variable to be stored into JSON Filename (required) string A path of the JSON file. If the file does not exist, it will be created. Write in File Modes combobox option a+ = file created if doesn’t exist and text appended at the end of the file, w+ = file created if it doesn’t exist but if it does, it get’s overwritten ## Normalize Dict to Dataframe ### Description Normalize semi-structured JSON data into a flat table. ### Parameters Parameter Type Description Dictionary (required) dict A dict type variable which will be JSON serialized and transformed into a DataFrame. Record Path (required) string Path in each object to list of records. New variable string A name of the new variable storing the created DataFrame. ## Database Select ### Description Execute database select on various databases. Supports one condition. For more advanced queries use DB Query. ### Parameters Common parameter Parameter Type Description From Comboentry Database table on which the query is executed. Select Comboentry Columns which should be returned by the query. Default to all columns as *. Column Comboentry The column on which the condition is evaluated. Operator Combobox option Condition operator. One of: =, <, <=, >, >=, <>, IN Value String entry A value against which the condition is evaluated. Limit String entry Number of records to return New variable String entry A name for the new Dataframe variable ## Database Insert ### Description Execute database insert on various databases. ### Parameters Common parameter Parameter Type Description Table name Comboentry Database table on which the query is executed. Dataframe Dataframe entry Dataframe which should be inserted to db table ## Database Delete ### Description Execute database delete on various databases. Supports one condition. For more advanced queries use DB Query. ### Parameters Common parameter Parameter Type Description From Comboentry Database table on which the query is executed. Column Comboentry The column on which the condition is evaluated. Operator Combobox option Condition operator. One of: =, <, <=, >, >=, <>, IN Value String entry A value against which the condition is evaluated. ## Database Update ### Description Execute database update on various databases. Supports one condition. For more advanced queries use DB Query. ### Parameters Common parameter Parameter Type Description Table name Comboentry Database table on which the query is executed. Column Comboentry The column in which the value is updated. Value String entry A value inserted to a given column in rows comforting a given condition Column Comboentry The column on which the condition is evaluated. Operator Combobox option Condition operator. One of: =, <, <=, >, >=, <>, IN Value String entry A value against which the condition is evaluated. ## Database Query ### Description Execute a given database query on various databases. ### Parameters Common parameter Parameter Type Description Table name Comboentry Database table on which the query is executed. New variable String entry If select query is executed the result will be stored into variable with this name ## Read PDF ### Description Loads a .PDF file as a new Forloop variable and, if desired, saves it’s contents as a new .txt file. ### Parameters Common parameter Parameter Type Description File name string Path to a PDF file, eg. /Users/GoodRead/Divina_Commedia_Alighieri.pdf. Variable name string A name of the new Forloop variable in which the pdf contents will be stored. Create .txt file checkbox (bool) If checked (True) a new text file with the name of the variable name will be created in the root folder. --- URL: https://www.forloop.ai/docs/cleaning Title: Cleaning Section: Node Catalogue --- # Cleaning ## Drop Column ### Description Drop Column Node drops selected column(s) in a data table. Hint For a detailed walkthrough see the step-by-step guide. ### Parameters The Drop Column Node requires three parameters, input dataframe, a column to be deleted and a name for a new variable. Dataframe entry input expects variable rectangle with Dataframe, input Columns can be selected from the combobox and new bariable expects string. Parameter Type Description Dataframe Dataframe entry Dataframe variable rectangle Column(s) Combobox option A name of the column or name of columns to be deleted. More than one column can be selected from the combobox. New variable String entry A name for the new Dataframe variable ### Step-by-step guide ## Rename Column ### Description Rename Column Node sets a new name (header) to the selected column. Hint For a detailed walkthrough see the step-by-step guide. ### Parameters The Rename Column Node requires 2 parameters (other than input dataframe and new dataframe name), a column whose name is to be changed and the new name. Parameter Type Description Dataframe Dataframe entry Dataframe variable rectangle Column Comboentry The column to be renamed. The name of the column can be either written or selected from the combobox. New name String entry A new name which will be used as a header for the selected column, e.g. “New_column_name”. New variable String entry A name for the new Dataframe variable ### Step-by-step guide ## Select Columns ### Description Select Column Node takes out selected column(s) from a data table. Hint For a detailed walkthrough see the step-by-step guide. ### Parameters The Select Column Node requires at least one parameter, a column(s) to be selected from the data table. Parameter Type Description Dataframe Dataframe entry Dataframe variable rectangle Columns to select Combobox option A name of the column or name of columns to be selected from the data. More than one column can be chosen in the combobox. New variable String entry A name for the new Dataframe variable ### Step-by-step guide ## Add Constant Column ### Description Add Constant Column Node creates a new constant column of length equal to that of the data frame. Hint For a detailed walkthrough see the step-by-step guide. ### Parameters Add Constant Column Node requires 2 parameters, a value, i.e. a number which will fill in the constant column, and a new column name. Parameter Type Description Dataframe Dataframe entry Dataframe variable rectangle Value Integer/Float entry A constant value filling the new column. Column name String entry A name (header) of the newly created constant column, e.g. “A_new_constant_column”. New variable String entry A name for the new Dataframe variable ### Step-by-step guide ## Round To Higher Frequency ## Column Math Operation (Outdated) ### Description Column Math Operation Node performs a mathemathical operations on the dataframes columns, e.g. sums values of two or more columns. ### Parameters The number of parameters is dependend on the chosen operation. The user can choose the operation in the first combobox which will trigger the revelation of other combobox(es). All operation contain the Result name parameter which defines the name of the resulting column. Parameter Type Description Choose math operation combobox A desired math operation. Result name string A name of the resulting column which will hold the values obtained from the performed operation. ## Search String (Outdated) ### Description Search String Node returns all occurences of strings satisfying given pattern. ### Parameters Search String Node requires 2 parameters: Parameter Type Description Column combobox The column selected for a string search. Pattern string A string pattern used for search. ## Replace String ### Description Replace String Node replaces all strings (or sub-strings) in selected column which either contain some pattern or exactly match it. ### Parameters Replace String Node requires 4 parameters: Parameter Type Description Dataframe Dataframe entry Dataframe variable rectangle Replace in columns Combobox option Column(s) selected for the string replacement. Match type Combobox option pattern → all strings containing the pattern will be replaced; exact → only string exactly matching the pattern will be replaced Replace substring Combobox option Enables to replace substrings. Pattern String entry The pattern used in the replacement process. Replacement String entry The string which will be replaced for all the selected values. New variable String entry A name for the new Dataframe variable ## Filter Data ### Description Takes a dataset and removes or selects the data that satisfies given expression. For example let’s say we have a phonebook of all employees working in an international company and we want to select only the contacts for those who work in Germany. Then we would pass an expression looking something like `country` == ‘Germany’. ### Parameters Filter Data Node requires at least 1 parameter: Parameter Type Description Dataframe Dataframe entry Dataframe variable rectangle Column(s) Combobox option Column(s) selected for the string filtering. Filter by string String entry Filter string which is to be filtered. Keep matched or drop Combobox option Set icon to either keep or drop rows satisfying the expression New variable String entry A name for the new Dataframe variable ## Split String ### Description Splits strings in a given column on a given substring and only keeps element on a given position of a resulting split list. ### Parameters Parameter Type Description Dataframe Dataframe entry Dataframe variable rectangle Column Combobox option Column selected for the string splitting. Split on String entry String on which the icon splits the strings in a given column. Select index String entry Index position of resulting split list, on which the result should be stored, starting from 0. Keep old column Combobox option Decide whether the old column is or be dropped or not New column String entry A name for the new Dataframe column New variable String entry A name for the new Dataframe variable Suppose we have a datetime column with dates in the dd/mm/yyyy format, then split on ‘/’ with select index 0 will give us dd value in a column named {new_column}. ## Sort Data ### Description Sorts selected columns in either ascending or descending order. ### Parameters Parameter Type Description Dataframe Dataframe entry Dataframe variable rectangle Sort column (2x) Comboentry Names of the column to be sorted, e.g. we have a dataframe containing columns Name, Surname, Age, Salary and want to sort it in ascending order by Age and Salary → we will enter: Age, Salary Ascending (2x) Checkbox Tick the checkbox for ascending sort. New variable String entry A name for the new Dataframe variable ## Detect Or Remove Outliers ### Description Detect or remove given number of percentage of outliers in a given column(s) ### Parameters Parameter Type Description Dataframe Dataframe entry Dataframe variable rectangle. Outlier mode Combobox option Decide whether to detect or remove outliers. Columns Combobox option Choose in which columns the outliers should be detected. Ratio as outliers Float entry Choose percentage of outliers to be found. Choose either ratio or top n. Top N as outliers Integer entry Choose number of outliers to be found. Choose either ratio or top n. New variable String entry A name for the new Dataframe variable. ## Remove Duplicates ### Description Removes all duplicate values in selected columns while keeping the first/last occurence or none. ### Parameters Parameter Type Description Dataframe Dataframe entry Dataframe variable rectangle. Keep value Combobox option Decide handler behaviour. Either keep only first, last, or none of outliers. Considered Columns Comboentry Choose in which columns the duplicates should be detected. New variable String entry A name for the new Dataframe variable. ## Remove Empty Rows ### Description Removes empty rows. If some ID columns are filled, but other columns are empty, the filled columns can be ignored. ### Parameters Parameter Type Description Dataframe Dataframe entry Dataframe variable rectangle. Mode Combobox option Decide whether to detect or remove empty rows. ID Columns Comboentry Choose in which columns the id values are filled (columns can be ignored). New variable String entry A name for the new Dataframe variable. ## Find Difference in Data ### Description Compare two dataframes with similar columns and return their difference. ### Parameters Parameter Type Description Dataframe Dataframe entry First dataframe variable rectangle. Subtract Dataframe Dataframe entry Variable rectangle storing subtracted dataframe. New variable String entry A name for the new Dataframe variable. ## Column Wise Shift ### Description Inspect two dataframe columns, shift their values so that corresponding values (e.g. name and domain) are located on the same row. Inserts empty cell or deletes filled cell where necessary. ### Parameters Parameter Type Description Dataframe Dataframe entry First dataframe variable rectangle. Mode Combobox option Choose whether rows that do not match should be removed or kept with the other column being shifted and inserted with nan Reference column Combobox option Choose the first column for comparison Incomplete column Combobox option Choose the second column for comparison New variable String entry A name for the new Dataframe variable. ## KNN Imputation ### Description Apply numeric KNN Imputation on selected column. ### Parameters Parameter Type Description Dataframe Dataframe entry Dataframe variable rectangle. Column Comboentry Choose in which column values should be imputed. New variable String entry A name for the new Dataframe variable. ## Imputation ### Description Apply numeric or categorical imputation on selected column. ### Parameters Parameter Type Description Dataframe Dataframe entry Dataframe variable rectangle. Imputed Column Comboentry Choose in which columns values should be imputed. Impute choice Comboentry Choose how the values should be generated. Either choose function (or zero) from combobox, or write value in entry. New variable String entry A name for the new Dataframe variable. ## Concatenate ### Description Concatenates values in the two selected dataframes into a new one. ### Parameters Common parameter Parameter Type Description Dataframe (2x) Dataframe entry Dataframe variable rectangle. Append Combobox option Choose whether rows or columns should be appended Join Combobox option Choose whether join executed should be inner or outer New variable String entry A name for the new Dataframe variable. ## Join Dataframes ### Description Joins two dataframes on (possibly muptiple) columns. ### Parameters Common parameter Parameter Type Description Dataframe (2x) Dataframe entry Dataframe variable rectangle. On columns Combobox option Choose columns on which join should be executed How Combobox option Choose join mode - left, right, outer, inner, cross New variable String entry A name for the new Dataframe variable. ## Apply Mapping ## Aggregate Groups ### Description Group (not required) data and execute numerical and/or categorical aggregations on given column(s) ### Parameters Common parameter Parameter Type Description Dataframe Dataframe entry Dataframe variable rectangle. Columns to Group by Combobox option Choose columns on which group by should be executed Columns to Aggregate Combobox option Choose columns on which group by should be executed Numeric aggregations Combobox option Choose numerical aggregations that should be executed on numerical columns - sum, mean, **median, max, min, count, mode Categorical aggregations Combobox option Choose categorical aggregations that should be executed on categorical columns - mode New variable String entry A name for the new Dataframe variable. --- URL: https://www.forloop.ai/docs/control Title: Control Section: Node Catalogue --- # Control ## New Pipeline ### Description Marks the starting point of a pipeline, i.e. it should be the first node in the pipeline. ## Finish Pipeline ### Description Marks the ending of a pipeline, i.e. it should be the last node in the pipeline. ## Add New Variable ### Description Add New Variable Node creates a new variable which gets stored in the variable explorer. Hint For a detailed walkthrough see the step-by-step guide. ### Parameters Add New Variable Node requires 2 parameters for a succesful creation of a new variable. Parameter Type Description Variable name string A name of the just defined variable under which will it be stored in the variable explorer, e.g. “Int1”. Value int/float/str/list/dict/bool Value of the new variable, e.g. [1, “b”, {“key”: “value”}]. ### Step-by-step guide ## Convert Variable Type ### Description Convert Variable Type Node converts the type of the selected variable to a different type if such conversion makes sense. Hint For a detailed walkthrough see the step-by-step guide. ### Parameters Convert Variable Type Node requires 2 parameters for a succesful conversion of the variable type. Parameter Type Description Variable name string A name of the variable present in the variable explorer whose type is to be changed, e.g. “List1”. Convert to type combobox A new type of the selected variable after the conversion. It can be selected as one of the options of the combobox. ### Step-by-step guide ## Math Modify Variable ### Description Math Modify Variable Node serves to perform a mathematical operations on variables in the variable explorer. Hint For a detailed walkthrough see the step-by-step guide. ### Parameters Math Modify Variable Node requires 3-4 parameters to succesfully perform a mathematical operation on a variable. The last parameter, New variable name, is optional in a sense that if left blank the value of the chosen variable will be rewritten adequately to the performed operation. However if a new name is inserted a new variable bearing the new name with the value of the old one corrected by the mathematical operation will be created while preserving the old variable. Parameter Type Description Variable name string A name of the variable present in the variable explorer which would be used for the operation. Math operation combobox A math operation to be perfomed on the selected variable. It can be selected as one of the options of the combobox. Argument int/float An argument of the selected mathematical operation, i.e. in case of e.g. division the argument is divisor. New name string Name of the new variable whose value will be equal to the old value modifed by the selected operation. If left blank the initial variable will get overwritten. ### Step-by-step guide ## String Modify Variable ### Description String Modify Variable Node serves to perform a various operations on variables of the string type. Hint For a detailed walkthrough see the step-by-step guide. ### Parameters String Modify Variable Node requires 3-5 parameters to succesfully perform a string operation on a variable. Argument 2 is in some cases optional, e.g. split(argument1) does not need a second argument. The last parameter, New variable name, is optional in a sense that if left blank the value of the chosen variable will be rewritten adequately to the performed operation. However if a new name is inserted a new variable bearing the new name with the value of the old one modified by the selected operation will be created while preserving the old variable. Parameter Type Description Variable name string A name of the variable present in the variable explorer which would be used for the operation. String operation combobox A string operation to be perfomed on the selected variable. It can be selected as one of the options of the combobox. Argument string The first argument of the selected string operation. Argument 2 string The second argument of the selected mathematical operation. New name string Name of the new variable whose value will be equal to the old value modifed by the selected operation. If left blank the initial variable will get overwritten. ### Step-by-step guide ## List Modify Variable ### Description List Modify Variable Node serves to perform a various operations on lists. Hint For a detailed walkthrough see the step-by-step guide. ### Parameters List Modify Variable Node requires 3-4 parameters to succesfully perform an operation on a stored list. The last parameter, New variable name, is optional in a sense that if left blank the value of the chosen variable will be rewritten adequately to the performed operation. However if a new name is inserted a new variable bearing the new name with the value of the old one modified by the selected operation will be created while preserving the old variable. Parameter Type Description Variable name string A name of the variable (list) present in the variable explorer which would be used for the operation. List operation combobox A string operation to be perfomed on the selected variable. It can be selected as one of the options of the combobox. Argument Get element, Remove, Pop – int // Append, Index – int/float/str/list/dict/bool // Concatenate – list An argument of the selected list operation. New name string Name of the new variable whose value will be equal to the old value modifed by the selected operation. If left blank the initial variable will get overwritten. ### Step-by-step guide ## Dictionary Modify Variable ### Description Dictionary Modify Variable Node serves to perform a various operations on dictionaries. Hint For a detailed walkthrough see the step-by-step guide. ### Parameters Dictionary Modify Variable Node requires 2-4 parameters to succesfully perform an operation on a stored dictionary. Argument is required only for the Get value by key function. The last parameter, New variable name, is optional in a sense that if left blank the value of the chosen variable will be rewritten adequately to the performed operation. However if a new name is inserted a new variable bearing the new name with the value of the old one modified by the selected operation will be created while preserving the old variable. Parameter Type Description Variable name string A name of the variable (dictionary) present in the variable explorer which would be used for the operation. Dictionary operation combobox A string operation to be perfomed on the selected variable. It can be selected as one of the options of the combobox. Argument 1 any A first argument of a given operation (can be left blank - get keys, get values). Argument 2 any A second argument of agiven operation (can be left blank). New name string Name of the new variable whose value will be equal to the old value modifed by the selected operation. If left blank the initial variable will get overwritten. ### Step-by-step guide ## If Condition ### Description Toggles one of the two channels depending on whether a condition defined by the user holds true or not. ### Parameters If Condition Node requires 3 parameters: Parameter Type Description Value 1 int/float/str/list/dict/bool The first value used in the condition. Operator combobox The operator used used in the condition. Value 2 int/float/str/list/dict/bool The second value used in the condition. Together they form a condition, e.g. column_name == ‘city’ which can be True or False. The Toggle button then switches the active channel, i.e. the user can choose if the “True branch” will run or the “False branch”. ## Print Variable ### Description Takes a variable as an input and prints it into a console. ### Parameters Parameter Type Description Variable name string A name of the variable to be printed. ## Load Python Script ### Description Loads a Python script specified by a path leading to a .py file. Optionally runs the script after loading. ### Parameters Parameter Type Description File path (required) string A path leading to a .py file. Load and run bool (checkbox) If True the script is run after loading, else load the file only. ## Load Jupyter Script ### Description Loads a Jupyter script specified by a path leading to a .ipynb file. Optionally runs the script after loading. ### Parameters Parameter Type Description File path (required) string A path leading to a .ipynb file. Load and run bool (checkbox) If True the script is run after loading, else load the file only. ## Define Function ### Description Creates a new function variable in Forloop. The function is defined via a Python code snippet containing the function, eg.: def my_func(arg1: str, arg2: str): print(f'Arg1 is {arg1}') print(f'Arg2 is {arg2}') ### Parameters Parameter Type Description Code string A Python function definition code snippet, eg. the one shown above. ## Define Lambda Function ### Description Creates a new function variable in Forloop. The function is defined similarly to the way in Python but in an easier to grasp manner. Let’s consider this lambda function as an example: my_awesome_lambda = lambda x,y,z,r: x**2 + y**2 + z**2 == r**2 Clearly it gives us an information whether or not a point with coordinates (x,y,z) is situated on the surface of a sphere of radius r. But when we read the code it can be naturally divided into three parts; a name, arguments and a code (what it does). And that is exactly how the entering parameters are structured. ### Parameters Parameter Type Description Name (req.) string A name of the lambda function. In the examplary case my_awesome_lambda. Arguments (req.) string Arguments of the lambda function. In the examplary case x,y,z,r. Return (req.) string What it ought to return, ie. the code part. In the examplary case x**2 + y**2 + z**2 == r**2. ## Run Function ### Description Takes a function variable, runs it and creates a new variable containing the return value of the function. ### Parameters Parameter Type Description Function (req.) function A function variable (a variable rectangle from variable explorer containing a function). Return variable string Name of a variable containing the return value. If left blank, none will be created (usefull if we expect return None situation). ## Datetime Now ### Description Takes the current time and date and stores it as a single datetime (float/int in case of timestamp option) value. ### Parameters Parameter Type Description New variable name (req.) string A name of the variable storing the datetime value, e.g. carpe_diem. Format combobox options Datetime variable format. Datetime – 2022-07-06 23:05:45.2345, timestamp – 1657058754. Add decimals checkbox (bool) If checked (True), timestamp will contain decimals so it will be of the type float. Otherwise it will be an integer. ## New Datetime ### Description Creates a new datetime format variable from a given date and (optionally) time. ### Parameters Parameter Type Description New variable name (req.) string A name of the variable storing the datetime value, e.g. my_birthday. Year (req.) int Date’s year. Month (req.) int A month, for obvious reasons limited to a set {1,2,3,…11,12}. Day (req.) int A day, again strictly limited by 1 and 31, sometimes a little less. Time parameters (Hour, Minute, Second, Microsecond) int Parameters specifying time of the entered date (optional - default: 00:00:00.00). ## Datetime Difference ### Description Takes two datetimes and calculates their difference in a given unit. Do you want to know how many microseconds are remaining till your birthday or your favorite band’s next concert in your city? Search no further, we have got you covered! ### Parameters Parameter Type Description Datetime 1 (req.) datetime/numerical Datetime variable serving as a subtrahend. Datetime 2 (req.) datetime/numerical Datetime variable serving as a minuend. Format combobox A datetime unit in which a result will be calculated. Add decimals bool (checkbox) Add/not decimals in the result. Usefull especially when working with precise measurements (lower seconds/microseconds). New variable name string A name of the variable storing the result. ## Datetime Value ### Description Takes a datetime variable and extracts a piece of datetime information from it. E.g.: let’s say we’ve a variable ‘2022-10-22 12:15:00.00’ and want to extract an hour from it. We’ll obviously receive 12, for a year then 2022, a day 22 and so on and so forth. ### Parameters Parameter Type Description Datetime variable (req.) datetime/numerical A datetime variable. Return combobox A unit which will get extracted from the entered datetime value, e.g. hour. New variable name string A name of the variable storing the result. ## Datetime To String ### Description Converts datetime variable to a string with a given format. ### Parameters Parameter Type Description Datetime variable (req.) datetime/numerical A datetime variable. Return combobox A return format. Be aware that some strange formatting may not work and thus won’t get converted. New variable name string A name of the variable storing the result. ## Datetime From String ### Description Converts a datetime string into datetime variable. Warning Some formats (strange ones) may not be recognized as a datetime format. ### Parameters Parameter Type Description Datetime string (req.) string A datetime string, e.g. 1.5.2020. New variable name string A name of the new datetime variable. ## Datetime Add Delta ### Description Takes a datetime variable and adds a datetime difference to it. E.g.: let’s say we want to know, what is the date after 56 weeks from now. Note The difference can be negative as well! When it is it points in the past. So if we add -56 weeks, we get the information what day it was 56 week ago. ### Parameters Parameter Type Description Datetime variable (req.) datetime/numerical A datetime variable. Shift to add (req.) numerical A shift to be added to the datetime variable. Can be negative (then points in the past)! Unit combobox Unit of the time shift. New variable name string A name of the new shifted datetime variable. ## Subset Data in a Batch Note Coming soon… ## Export Data to Jupyter Script Note Coming soon… ## Train Model Note Coming soon… ## Predict Model Values Note Coming soon… --- URL: https://www.forloop.ai/docs/integrations Title: Integrations Section: Node Catalogue --- # Integrations ## New Google Sheet ### Description New Google Sheet Node serves for creation of a new Google sheet on a specified Google account. It requires two entries: users e-mail address and a name which will be given to the newly created sheet. Hint For a detailed walkthrough see the step-by-step guide . ### Parameters In order to succesfully create a new Google spreadsheet, two parameters are required as user entries. Parameter Type Description Sheet name string The entered string will be used as a name for the newly created spreadsheet, e.g. Test_sheet. E-mail string E-mail address with an access to Google worspace, e.g. john doe gmail com. The new spreadsheet will be created in spreadsheets on this account. ### Step-by-step guide The New Google Sheet Node is located in the INTEGRATIONS submenu of the BUILD menu. In order to activate it, click on the INTEGRATIONS button on the left panel… and then on the New Google Sheet option in the pop-up menu… A new icon labeled NewGoogleSheet will appear in the working space. After a click/tap on this icon two entry boxes will appear in the upper right corner as shown below. The first entry requires an e-mail address to which the newly created Google sheet will be paired and the second one stores the name of the sheet. Important E-mail address must be hosted by Google! Finally click on the Execute button to finish the creation of your new Google sheet. That’s it! Note It is advisable to check your Google account for the newly created sheet so as to avoid any inconvenience during the follow up work. ## Load Google Sheet ### Description Load Google Sheet Node serves, as the name suggests, for loading a Google sheet into Forloop. It requires three entries: Google file ID, sheet name and new variable name. It then creates a new Forloop object with an interactive icon which can be used for further analysis in the same manner as a data table. Hint For a detailed walkthrough see the step-by-step guide. ### Parameters In order to succesfully load a Google spreadsheet, three parameters are required as user entries. Parameter Type Description Sheet ID string Google file ID is a unique key defining the specific spreadsheet. It can be found in sheets URl, e.g.: https://docs.google.com/spreadsheets/d/ 1FApy2bGcFFmpg-lTNS8HWpq-fpHlGcJhvq-DXhr4b1o /edit#gid=0 Sheet name string The name of the loaded spreadsheet. It is essential to write the name in its full form, i.e. if the name is ‘test_sheet’ then writing ‘test sheet’ will not work. New variable name string A name that will be used for the newly created icon of the loaded spreadsheet. Therefore this field required an arbitrary string. ### Step-by-step guide The Load Google Sheet handler is located in the INTEGRATIONS submenu of the BUILD menu. In order to activate it, click on the INTEGRATIONS button on the left panel… and then on the Load Google Sheet option in the pop-up menu… A new icon labeled LoadGoogleSheet will appear in the working space. After a click/tap on this icon two entry boxes will appear in the upper right corner as shown below. The first entry requires the Google file ID of the sheet to be loaded and the second one the sheets name. Important Google file ID is a unique key defining the specific spreadsheet. It can be found in sheets URl, e.g.: https://docs.google.com/spreadsheets/d/ 1FApy2bGcFFmpg-lTNS8HWpq-fpHlGcJhvq-DXhr4b1o /edit#gid=0 After filling in both entry boxes and clicking the Execute button beneath them… a new icon named as the original spreadsheet will appear in the work space. This icon serves as a gateway to the object containing the loaded spreadsheet so it can be handled in the similar manner as e.g. loaded Excel file or other data table object. For example the data stored in the sheet can be viewed as table by using Data Grid View. ## Copy Google Sheet ### Description Copy Google Sheet Node makes a copy of the spreadsheet identified by Sheet ID for the Google account belonging to the E-mail address. An arbitrary name can be chosen for such copy. Hint For a detailed walkthrough see the step-by-step guide. ### Parameters In order to succesfully load a Google spreadsheet, three parameters are required as user entries. Parameter Type Description Sheet ID string Google file ID is a unique key defining the specific spreadsheet. It can be found in sheets URl, e.g.: https://docs.google.com/spreadsheets/d/ 1FApy2bGcFFmpg-lTNS8HWpq-fpHlGcJhvq-DXhr4b1o /edit#gid=0 Sheet copy name string The entered string will be used as a name for the newly created copy of the original spreadsheet, e.g. Copied_sheet1. E-mail string E-mail address with an access to Google worspace, e.g. john doe gmail com. The copy of the original spreadsheet will be created in spreadsheets on this account. ### Step-by-step guide ## Update Cell ### Description Update Cell Node can be used for entering new data in a specific single cell of a spreadsheet or altering already existing data in such a cell. Hint For a detailed walkthrough see the step-by-step guide. ### Parameters Update Cell Node requires 4 parameters as user entries, all of which must be filled in. Parameter Type Description Sheet ID string Google file ID is a unique key defining the specific spreadsheet. It can be found in sheets URl, e.g.: https://docs.google.com/spreadsheets/d/ 1FApy2bGcFFmpg-lTNS8HWpq-fpHlGcJhvq-DXhr4b1o /edit#gid=0 Sheet name string The name of the spreadsheet, whose cell is going to be updated. It is essential to write the name in its full form, i.e. if the name is ‘test_sheet’ then writing ‘test sheet’ will not work. Cell name string Specifies which cell will get updated, e.g. ‘B4’. Value int/float/list/dict/str/bool Data to be inserted into the specified cell. ### Step-by-step guide ## Delete Sheet Row ### Description Delete Sheet Row Node serves to deletion of a single row or a series of rows in a Google spreadsheet. Hint For a detailed walkthrough see the step-by-step guide. ### Parameters Delete Sheet Row Node requires 3 parameters (Sheet ID, Sheet name, Start row number) to delete a single row and 4 parameters (Sheet ID, Sheet name, Start row number, Stop row number) to delete a series of rows. Parameter Type Description Sheet ID string Google file ID is a unique key defining the specific spreadsheet. It can be found in sheets URl, e.g.: https://docs.google.com/spreadsheets/d/ 1FApy2bGcFFmpg-lTNS8HWpq-fpHlGcJhvq-DXhr4b1o /edit#gid=0 Sheet name string The name of the spreadsheet, whose row(s) is/are going to be deleted. It is essential to write the name in its full form, i.e. if the name is ‘test_sheet’ then writing ‘test sheet’ will not work. Start row number integer A number of the row to be deleted or the initial row of the series to be deleted. The numbering preserves the Python logic, ie. the first row corresponds to number 0! Stop row number integer A number of the last row of the series of rows to be deleted. If left blank, only a single row, i.e. row no. Start row number + 1 (Python logic), will be deleted. ### Step-by-step guide ## Parse Data To Sheet ### Description Parse Data To Sheet Node enables us to insert data from a text file (.txt or .csv) into a spreadsheet. Hint For a detailed walkthrough see the step-by-step guide. ### Parameters Parse Data To Sheet Node requires at least 3 parameters (Filename, Sheet ID, Sheet name) to succesfully insert the data into the specified spreadsheet. Parameter Type Description Filename string A path to the data file must be filled in into the Filename entry either manually or via file explorer by clicking the Look up file button below the entry line. Sheet ID string Google file ID is a unique key defining the specific spreadsheet. It can be found in sheets URl, e.g.: https://docs.google.com/spreadsheets/d/ 1FApy2bGcFFmpg-lTNS8HWpq-fpHlGcJhvq-DXhr4b1o /edit#gid=0 Sheet name string The name of the spreadsheet into which should the data be loaded. It is essential to write the name in its full form, i.e. if the name is ‘test_sheet’ then writing ‘test sheet’ will not work. Delimiter string The delimiter of the data file. For example for a .csv file the input should be “,” without the quotation marks. If left blank a .txt file separated with a space is expected. Starting cell string/integer Starting point for the parsed data, i.e. this cell will be an upper left corner of the inserted data. If left blank the “A1” cell will be the starting point. ### Step-by-step guide ## Pipedrive Connect ### Description Pipedrive Connect Node creates the connection with the pipedrive specified by the entered arguments, i.e. Company domain name and Pipedrive token. Hint For a detailed walkthrough see the step-by-step guide. ### Parameters Pipedrive Connect Node requires at 2 parameters to make a successful connection. Parameter Type Description Company domain name string A name of the company domain connecting to the pipedrive, e.g. forloop. Pipedrive token string/integer A special key required for authentification before the connection to the pipedrive. En example of such token: ah0jjaks3masjas3mamd0bre. ### Step-by-step guide ## Pipedrive Get Stages ### Description Pipedrive Get Stages Node gets stages from the connected pipedrive and saves them as a new variable. Hint For a detailed walkthrough see the step-by-step guide. ### Parameters Parameter Type Description Save as string A name of the newly created variable containing the data from the pipedrive, e.g. ‘stages_dict_1’. ### Step-by-step guide ## Pipedrive Get Users ### Description Pipedrive Get Users Node retrieves the list of users from the connected pipedrive and saves it as a new variable. Hint For a detailed walkthrough see the step-by-step guide. ### Parameters Parameter Type Description Save as string A name of the newly created variable containing the data from the pipedrive, e.g. ‘users_dict_1’. ### Step-by-step guide ## Pipedrive Add Person ### Description Pipedrive Add Person Node inserts a new user data into the connected pipedrive. Hint For a detailed walkthrough see the step-by-step guide. ### Parameters Parameter Type Description Name string The name of the new user, e.g. ‘Patrick Jane’. E-mail string New users e-mail address, e.g. red john forloop ai Phone number string/integer New users phone number, e.g. +420111222333. ### Step-by-step guide ## Pipedrive Add Deal ### Description Pipedrive Add Deal serves to enter a deal (basically a post request) to the connected pipedrive. ### Parameters Pipedrive Add Deal Node requires 3 parameters which form a payload of the post request: Parameter Type Description Title string The title of the entered deal, e.g. ‘Damn goog deal!’ Stage ID string/integer An id code of the deal used for further identification. User ID string/integer An id of the user inserting the deal. ## Pipedrive Delete Deal ### Description Pipedrive Delete Deal Node deletes the deal specified by the Deal ID from the connected pipedrive. Hint For a detailed walkthrough see the step-by-step guide. ### Parameters Parameter Type Description Deal ID string/integer An id code of the deal to be deleted. ### Step-by-step guide ## Pipedrive Add Note ### Description Pipedrive Add Note serves to enter a note to the deal in the pipedrive specified by its Deal ID. Hint For a detailed walkthrough see the step-by-step guide. ### Parameters Parameter Type Description Content string The content which will be added as a note to the specified deal, e.g. “A deal is never late, nor is it early, it arrives precisely when it means to.” Deal ID string/integer An id code of the deal to which the note will be connected. ### Step-by-step guide ## Airtable Connect ### Description Airtable Connect Node establishes a connection to the specified Base on https://airtable.com. Hint For a detailed walkthrough see the step-by-step guide. ### Parameters Parameter Type Description API key string A 17 character string containing letters and numbers which enables the usage of the API for a specific account on https://airtable.com. Base ID string A 17 character string containing letters and numbers that specifies a Base ### Step-by-step guide ## Airtable Add Record ### Description Adds a record (a row of data) to the connected AirTable. ### Parameters Airtable Add Record requires 2 parameters: Parameter Type Description Table name string A name of the AirTable table in which the entry will be inserted. Value dictionary A dictionary containing the AirTable tables column names as keys and entered values as dictionary values, e.g. {‘Name’:’Gandalf’,’Surname’:’Stormcrow’,’Age’:24000, ‘Occupation’:’Wizard’} ## Airtable Parse Data ### Description AirTable Parse Data either appends the data to the selected Airtable or overwrites it, i.e. deletes all the data already present and appends the new data. ### Parameters 3 parameters are required: Parameter Type Description Table name string A name of the AirTable table in which the entry will be inserted. Filename string A path to the .csv file containing the data to be loaded, e.g. /Users/lord_voldemort/Desktop/my_evil_data.csv. Operation combobox Append → the data from the loaded file will be added to the end of the table; Overwrite all data in the table will be overwritten (Be careful as this method might lead to a data loss!). ## Airtable Read Table ### Description Reads the Airtable specified by its name (and the API key and base ID provided to the connection node) and saves its contents as a new variable in the variable explorer in the form of the list of dictionaries. ### Parameters 2 parameters are required: Parameter Type Description Table name string A name of the AirTable table to be read. Save as string A name of the new variable which will contain the data from the Airtable, e.g. my_lovely_new_var. --- URL: https://www.forloop.ai/docs/custom_nodes Title: Custom Section: Node Catalogue --- # Custom --- URL: https://www.forloop.ai/docs/pipelines_content_table Title: Pipelines Section: Content tables --- # Pipelines The Pipelines Content Table is your go-to hub for managing your pipelines. It’s like a control center for all things pipeline-related. From this table, you can run, load, share, or delete pipelines with just a few clicks. It’s a convenient and centralized location where you can easily access and manage your pipeline assets. Consider it your command center for pipeline management, right at your fingertips! --- URL: https://www.forloop.ai/docs/scripts_content_table Title: Scripts (Beta) Section: Content tables --- # Scripts (Beta) The Script Content Table is your one-stop destination for basic script control operations. It’s a convenient place where you can create, show, and delete scripts with ease. Think of it as a gallery showcasing all your active scripts, neatly organized in one place. Whether you need to create a new script, view the details of an existing one, or bid farewell to a script that has served its purpose, the Script Content Table has got you covered. It’s your command center for script management and a handy reference for all your script-related endeavors. --- URL: https://www.forloop.ai/docs/datasets_content_table Title: Datasets (Beta) Section: Content tables --- # Datasets (Beta) Note Coming soon… --- URL: https://www.forloop.ai/docs/databases_content_table Title: Databases (Beta) Section: Content tables --- # Databases (Beta) The Database Content Table is your one-stop shop for working with database connections in Forloop. It encapsulates the core functionalities needed to manage your database connections effectively. Here’s what it offers: - Add new database connection: Easily create and set up new connections to your databases. - Get all tables of a database: Retrieve a comprehensive list of tables within a specific database. - Edit database connection: Make modifications or updates to your existing database connections. - Test database connection: Verify the connectivity and functionality of your database connections. - Remove database connection: Bid farewell to database connections that are no longer needed. Additionally, the Database Content Table serves as a gallery, showcasing all your active database connections. It provides a centralized view of your database connections, making it easy to manage and access them whenever needed. --- URL: https://www.forloop.ai/docs/machines_content_table Title: Machines (Beta) Section: Content tables --- # Machines (Beta) Note Coming soon… --- URL: https://www.forloop.ai/docs/code_view Title: Code View Section: View modes --- # Code View --- URL: https://www.forloop.ai/docs/data_grid_view Title: Data Grid View Section: View modes --- # Data Grid View --- URL: https://www.forloop.ai/docs/database_view Title: Database View Section: View modes --- # Database View --- URL: https://www.forloop.ai/docs/browser_view Title: Browser View Section: View modes --- # Browser View Browser view shows the screenshot of currently scraped page with highlighted elements (tables, bullet lists etc.) from icon ScanWebPage. Also allows to: - Select element and pass its XPath to icons: ClickXPath, ExtractXPath, ExtractTableXPath - Export the content of selected element (tables to .xlsx, bullet lists to .txt) - Export the content of all highlighted elements Works with (Show browser = False) in icon OpenBrowser as well. --- URL: https://www.forloop.ai/docs/add_database_connection Title: User Guide: Adding a Database Connection Section: Databases --- # User Guide: Adding a Database Connection This user guide provides instructions on how to add a database connection to our platform. There are two main methods available. ## Method 1: Using the “Data Sources” Menu Navigate to the main dashboard of the platform and from the BUILD menu, select Data Sources. In the expanded Data Sources menu, click on Add Database. This will open a pop-up form that allows you to either choose from already connected databases or add a new database connection. Clicking new database will lead you to a database connection form. ## Method 2: Using the “Add Connection” Button in the DATABASES Content Table Navigate to the main dashboard of the platform and select the DATABASES Content table. On the upper right corner of the content table, click on the “Add Connection” button. This will open a database connection form. ## Filling Out the Database Connection Form Regardless of which method you used to connect a new database you will end up facing a following form: To fill in this form, follow this guide: - Server: Input the host of your database. This will usually be an IP address or a domain name. - Database: Input the name of the database to which you wish to connect. - Port: Input the port that your database uses. - Username: Enter the username that you use for accessing your database. - Password: Enter the password that you use for accessing your database. - Local: Check this box if the database is located on your local machine. - Create new DB: Check this box if you want to create a new database schema with the specified name. - SQL Dialect: Select the appropriate database dialect from the provided combobox. - Finally, click on the “Submit” button to add the connection. Warning This process will add the database connection to the platform without checking the validity of the database details. --- URL: https://www.forloop.ai/docs/database_dialects Title: Supported Database Dialects Section: Databases --- # Supported Database Dialects Our platform supports multiple database dialects to meet the diverse requirements of your projects. Here is a list of the currently supported dialects: ## MySQL MySQL is an open-source relational database management system (RDBMS). It uses a client-server model and Structured Query Language (SQL) to interact with the database. ## SQL Server SQL Server is a relational database management system (RDBMS) developed by Microsoft. It supports a wide range of transaction processing, business intelligence, and analytics applications in corporate IT environments. ## PostgreSQL PostgreSQL is a powerful, open-source object-relational database system. It uses and extends the SQL language and is known for its proven architecture, reliability, data integrity, robust feature set, extensibility, and the dedication of the open-source community behind the software. ## Xlsx Structure Our platform also supports reading data from Excel (Xlsx) file structures. This allows for the importing of tabular data from spreadsheets, which can then be queried as if they were structured databases. ## MongoDB MongoDB is a source-available cross-platform document-oriented database program. Classified as a NoSQL database program, MongoDB uses JSON-like documents with optional schemas. It is known for its flexibility and scalability, making it suitable for a wide range of applications. Note When adding a database connection, remember to select the correct database dialect from the “Database Dialect” combobox. --- URL: https://www.forloop.ai/docs/database_operations Title: Working with Databases Guide Section: Databases --- # Working with Databases Guide This guide provides instructions on how to work with databases in our platform. There are several methods to manage and manipulate your databases. ## Method 1: Content Tables - Navigate to the main dashboard of the platform. - From the left-hand side menu under the “Manage” section, select “DATABASES”. - This will show a list of all connected databases. You’ll see the database name, type (dialect), and four methods for each database: - Get: Retrieves all tables from the selected database and creates a dark box for each table in the platform. These boxes can be used for further operations like select, update, insert, delete, etc. - Edit: Opens a new form with prefilled database connection details. You can change any field and press “Submit” to update the connection details. - Test: Tests the connection to the selected database. - Remove: Removes the connection to the selected database. ## Method 2: Using Database Node - From the left-hand side menu under the “Build” section, select “DATA SOURCES”. - Then, select “Add Database”. This will create a new database node which can be used for further operations. ## Method 3: Using Database Operation Nodes - In the left-hand side menu under the “Build” -> “DATA SOURCES” section, you will find nodes called “DB Select”, “DB Insert”, “DB Update”, “DB Delete”, “DB Query”, etc. - Click on the node based on the operation you want to perform. For example, if you want to select from a database, click “DB Select”. This will create a “DB Select” node and open a form on the right side of the platform. - Database: Choose the database from which to select data. - From: Choose the table from which to select data. To select all columns, use the “*”. - Select: Enter the columns to select, separated by commas. - Where section: Enter the conditions for the operation. - Column: Enter the column for the WHERE condition. - Operator: Select the operator for the WHERE condition (e.g., <, >, =). - Value: Enter the value for the WHERE condition. The query will select rows where the column “operator” value condition is met. - Limit: Enter the maximum number of rows to retrieve. - New Variable: Enter the name of the new variable. This will hold the selected dataframe. --- URL: https://www.forloop.ai/docs/wizard Title: Wizard Section: Analyze --- # Wizard Imagine having a trusty wizard by your side during your mighty quest! While Forloop Wizard may not be able to conjure fireballs or cast magical spells, it possesses incredible data magic powers. This wizard is here to assist you with your data-related challenges! The Forloop Wizard can perform various enchantments on your data, including: - Detecting missing values: The wizard can identify those sneaky missing values in your datasets, helping you uncover and handle them effectively. - Finding outliers: With a flick of its virtual wand, the wizard can spot outliers in your data, allowing you to investigate and address them accordingly. - Predicting next steps: The wizard can use its mystical knowledge to suggest the next steps in your data cleaning process. It’s like having a wise guide who can foresee the most suitable actions to take. So, while the Forloop Wizard may not possess traditional magical abilities, its data prowess and analytical skills make it an invaluable ally in your data exploration and preparation journey. Harness the power of the wizard and let it work its data magic for you! ## Automatic dataframe cleaning Forloop platform analyzes dataframe and recommends possible cleaning steps. ## Pipeline builder recommendations Forloop platform analyzes pipeline created so far and recommends further nodes. ## Data categories predictions Forloop platform analyzes imported Dataframes and predicts category of each column. Currently, the supported categories are following: text, number, date, price, telephone, address, domain, email, name, unknown ## Possible joins finder Forloop platform evaluates overlap in columns associated with similar category (see Data categories predictions). It records dataframe containing column combinations whose overlap in both directions (values in A being in B and the other way around) exceeds 40 %. Other than that, we also record column name similarities and table A name being contained in the column B name. --- URL: https://www.forloop.ai/docs/monitoring Title: Monitoring Section: Analyze --- # Monitoring Note Coming soon… --- URL: https://www.forloop.ai/docs/triggers Title: Triggers Section: Orchestration --- # Triggers Note Coming soon… --- URL: https://www.forloop.ai/docs/mainpage Title: Forloop.ai main page Section: Help --- # Forloop.ai main page Important For further information please visit our website https://forloop.ai/. --- URL: https://www.forloop.ai/docs/contact_us Title: Contact us Section: Help --- # Contact us Important For further assistance please contact us at contact forloop ai. ## Site pages --- URL: https://www.forloop.ai Title: Self-healing web scraping data feeds | Forloop --- Build and deploy web scrapers, APIs and data workflows. Let Forloop autonomously maintain them. --- URL: https://www.forloop.ai/access Title: Access Platform : : Forloop --- Forloop is the production-ready no-code platform for external data process automation. --- URL: https://www.forloop.ai/app/bonnier Title: Bonnier : : Forloop --- Forloop is the production-ready no-code platform for external data process automation. --- URL: https://www.forloop.ai/blog Title: Blog : : Forloop --- Forloop is an External Data Platform - enabling data teams smoothly integrate and leverage external data as part of their data stack, to have an information advantage on their markets. --- URL: https://www.forloop.ai/blog-categories/consumer-brands Title: Consumer Brands : : Forloop --- consumer-brands --- URL: https://www.forloop.ai/blog-categories/logistics Title: Logistics : : Forloop --- logistics --- URL: https://www.forloop.ai/blog-categories/product Title: Product : : Forloop --- product --- URL: https://www.forloop.ai/blog-categories/real-estate Title: Real Estate : : Forloop --- real-estate --- URL: https://www.forloop.ai/blog-categories/tutorial Title: Tutorial : : Forloop --- tutorial --- URL: https://www.forloop.ai/blog/ai-real-estate-crowdfunding Title: The Role of AI in Real Estate Crowdfunding : : Forloop --- ai-real-estate-crowdfunding --- URL: https://www.forloop.ai/blog/beautiful-soup Title: Beautiful Soup introduction: A web scraping tools in practice : : Forloop --- beautiful-soup --- URL: https://www.forloop.ai/blog/collect-data Title: How to collect data from any website in just a few minutes? : : Forloop --- collect-data --- URL: https://www.forloop.ai/blog/content-scraping-real-estate Title: Content scraping in real estate market - a general overview. : : Forloop --- content-scraping-real-estate --- URL: https://www.forloop.ai/blog/external-data Title: What is external data, and why it's crucial in 2023? : : Forloop --- external-data --- URL: https://www.forloop.ai/blog/external-data-privacy Title: The Importance of External Data Privacy and Security : : Forloop --- external-data-privacy --- URL: https://www.forloop.ai/blog/internal-and-external-data Title: Internal and External Data: What's the Difference and Why It Matters : : Forloop --- internal-and-external-data --- URL: https://www.forloop.ai/blog/lease-office-space Title: How ShareSpace captured the UK and German market cost-efficiently, with Forloop. : : Forloop --- lease-office-space --- URL: https://www.forloop.ai/blog/market-intelligence Title: Introduction to Market Intelligence : : Forloop --- market-intelligence --- URL: https://www.forloop.ai/blog/ml-in-freight Title: Machine Learning at Work - in Freight, Air cargo and Logistics : : Forloop --- ml-in-freight --- URL: https://www.forloop.ai/blog/nicotine-pouches-e-com-report---may-2024 Title: Nicotine Pouches E-commerce Report - May 2024 : : Forloop --- nicotine-pouches-e-com-report---may-2024 --- URL: https://www.forloop.ai/blog/non-accredited-investors Title: How to Start Investing as a Non-Accredited Investors in Real Estate Crowdfunding Platforms : : Forloop --- non-accredited-investors --- URL: https://www.forloop.ai/blog/playwright Title: Playwright introduction: A web scraping tools in practice : : Forloop --- playwright --- URL: https://www.forloop.ai/blog/productive-teams Title: Why we need more productive data teams? : : Forloop --- productive-teams --- URL: https://www.forloop.ai/blog/puppeteer Title: Puppeteer Python API introduction: A web scraping tools in practice : : Forloop --- puppeteer --- URL: https://www.forloop.ai/blog/real-estate-crowdfunding Title: Data-Driven Real Estate Crowdfunding: The Future of Property Investing : : Forloop --- real-estate-crowdfunding --- URL: https://www.forloop.ai/blog/real-estate-crowdfunding-challenges Title: Challenges and Problems in Real Estate Crowdfunding : : Forloop --- real-estate-crowdfunding-challenges --- URL: https://www.forloop.ai/blog/real-estate-crowdfunding-platform Title: Building a Successful Real Estate Crowdfunding Platform: Key Features and Tech : : Forloop --- real-estate-crowdfunding-platform --- URL: https://www.forloop.ai/blog/real-estate-crowdfunding-platform-leads-generation Title: Real Estate Crowdfunding platform leads generation strategies. : : Forloop --- real-estate-crowdfunding-platform-leads-generation --- URL: https://www.forloop.ai/blog/real-estate-crowdfunding-trends Title: The Future of Real Estate Crowdfunding: Market Trends and Opportunities : : Forloop --- real-estate-crowdfunding-trends --- URL: https://www.forloop.ai/blog/real-estate-regulatory-challenges Title: Navigating Regulatory Challenges in Real Estate Crowdfunding : : Forloop --- real-estate-regulatory-challenges --- URL: https://www.forloop.ai/blog/real-estate-use-cases Title: A curated list of 20 real estate use cases for external data : : Forloop --- real-estate-use-cases --- URL: https://www.forloop.ai/blog/scraping-best-practices Title: This is the Data: A Mandalorian Guide to Web Scraping Best Practices : : Forloop --- scraping-best-practices --- URL: https://www.forloop.ai/blog/scrapy Title: Scrapy introduction: A web scraping tools in practice : : Forloop --- scrapy --- URL: https://www.forloop.ai/blog/selenium Title: Selenium introduction: A web scraping tools in practice : : Forloop --- selenium --- URL: https://www.forloop.ai/blog/top-5-open-source Title: Top 5 Open-Source Web Scraping Tools in 2023 : : Forloop --- top-5-open-source --- URL: https://www.forloop.ai/blog/trade-marketing Title: Unleashing the Potential of AI in Trade Marketing in FMCG : : Forloop --- trade-marketing --- URL: https://www.forloop.ai/blog/web-scraping Title: A web scraping quick guide with a hands-on tutorial : : Forloop --- web-scraping --- URL: https://www.forloop.ai/blog/web-scraping-best-practices Title: Web scraping best practices gained from industry experience : : Forloop --- web-scraping-best-practices --- URL: https://www.forloop.ai/careers Title: Careers : : Forloop --- Forloop is an External Data Platform - enabling data teams smoothly integrate and leverage external data as part of their data stack, to have an information advantage on their markets. --- URL: https://www.forloop.ai/case-studies Title: Case Studies : : Forloop --- Check how exciting and challenging a journey with Forloop can be through the voices of our clients and partners. --- URL: https://www.forloop.ai/case-studies/sharespace Title: Real Estate Case Study with Sharespace : : Forloop --- Sharespace is a cloud platform that compare and lease office space with or without advisors help. --- URL: https://www.forloop.ai/data-collector Title: Data Collector : : Forloop --- Forloop Data Collector allows you to access and integrate External Data. Turn any website to an API, or access pre-made integrations with maps, weather, foot traffic data, or other 3rd party sources. --- URL: https://www.forloop.ai/data-scientist Title: Data Scientist : : Forloop --- Forloop’s self-learning system allows Data Scientists decrease time spent on data preparation by 40%. --- URL: https://www.forloop.ai/documentation-forloop Title: Documentation : : Forloop --- Documentation + API description for Forloop.ai platform --- URL: https://www.forloop.ai/fmcg Title: FMCG : : Forloop --- Forloop for the FMCG market. --- URL: https://www.forloop.ai/fmcg-copy Title: FMCG : : Forloop --- Forloop for the FMCG market. --- URL: https://www.forloop.ai/fmcg-copy-2 Title: FMCG : : Forloop --- Forloop for the FMCG market. --- URL: https://www.forloop.ai/intelligence Title: Data Preparation Intelligence : : Forloop --- Forloop Data Preparation Intelligence focuses on the data preparation process. It allows you to automate the whole process of data cleaning, joining, and transforming before they go to the model. --- URL: https://www.forloop.ai/logistics Title: Logistics : : Forloop --- Unify the freight chain data to improve pricing, matching, and CO2 tracking. --- URL: https://www.forloop.ai/mission Title: Mission : : Forloop --- Forloop is an External Data Platform - enabling data teams smoothly integrate and leverage external data as part of their data stack, to have an information advantage on their markets. --- URL: https://www.forloop.ai/no-code Title: No-code Pipeline Builder : : Forloop --- Forloop No-code Pipeline Builder focuses on data pipelines. It allows you to automate processes related to data science and data analytics. --- URL: https://www.forloop.ai/no-code-platform-for-web-scraping-and-data-automation Title: No-code platform for web scraping and data automation --- Forloop.ai is a no-code platform for web scraping and data preparation, empowering teams to rapidly gather, prepare, and automate data processes. --- URL: https://www.forloop.ai/pricing Title: Pricing : : Forloop --- Forloop is an External Data Platform - enabling data teams smoothly integrate and leverage external data as part of their data stack, to have an information advantage on their markets. --- URL: https://www.forloop.ai/pricing-v2 Title: Pricing : : Forloop --- Forloop is an External Data Platform - enabling data teams smoothly integrate and leverage external data as part of their data stack, to have an information advantage on their markets. --- URL: https://www.forloop.ai/pricing-v3-backup-forloop Title: Pricing v3 (backup) : : Forloop --- Forloop is an External Data Platform - enabling data teams smoothly integrate and leverage external data as part of their data stack, to have an information advantage on their markets. --- URL: https://www.forloop.ai/pricing-v4-backup-forloop Title: Pricing v4 (backup) : : Forloop --- Forloop is an External Data Platform - enabling data teams smoothly integrate and leverage external data as part of their data stack, to have an information advantage on their markets. --- URL: https://www.forloop.ai/real-estate Title: Real Estate : : Forloop --- Forloop for the real estate industry. --- URL: https://www.forloop.ai/team Title: Mission : : Forloop --- Forloop is an External Data Platform - enabling data teams smoothly integrate and leverage external data as part of their data stack, to have an information advantage on their markets. --- URL: https://www.forloop.ai/untitled Title: Untitled --- Untitled --- URL: https://www.forloop.ai/video-fmcg Title: FMCG : : Video --- Forloop for the FMCG market. --- URL: https://www.forloop.ai/webinars Title: Webinars : : Forloop --- Every two weeks we run live webinars where we cover topics related to data-driven organizations. --- URL: https://www.forloop.ai/webinars/external-data-in-real-estate Title: External data in Real Estate : : Forloop --- external-data-in-real-estate --- URL: https://www.forloop.ai/webinars/external-data-strategy Title: Why should external data be part of your data strategy? : : Forloop --- external-data-strategy