node-red-contrib-getdata-website

How to Install

Run the following command in the root directory of your Node-RED install

Usage

Get html from url: html is parsed from the GET URL request.

Mapping: A node to scrape html of msg.payload to a JSON. The transformation is defined by mapping property. This property is an object containing the scraping information. If you want to scrape a list, you have to use the listItem selector:

- `listItem` (String): The list item selector.
- `data` (Object): The fields to include in the list objects:
    - `<fieldName>` (Object|String): The selector or an object containing:
      - `selector` (String): The selector.
      - `convert` (Function): An optional function to change the value.
      - `how` (Function|String): A function or function name to access the
        value.
      - `attr` (String): If provided, the value will be taken based on
        the attribute name.
      - `trim` (Boolean): If `false`, the value will *not* be trimmed
        (default: `true`).
      - `closest` (String): If provided, returns the first ancestor of
        the given element.
      - `eq` (Number): If provided, it will select the *nth* element.
      - `texteq` (Number): If provided, it will select the *nth* direct text child.
        Deep text child selection is not possible yet.
        Overwrites the `how` key.
      - `listItem` (Object): An object, keeping the recursive schema of
        the `listItem` object. This can be used to create nested lists.

For the format of the selector, please refer to the Selectors section of the Cheerio library

How to selector css? https://www.w3schools.com/CSS/css_selectors.asp

EX: URL: https://www.americamagazine.org/

MAPPING: { "News": { "listItem": ".views-row", "data": { "Title": ".title.titleKarlaBold", "By-AuThor": ".byAuthor.titleKarlaBold" } } }