dynamic-render

Dynamic Render

Optimizes SEO by dynamically rendering javascript powered websites

Guide

• Install
• Getting Started
• Render Cycle
• Interceptors
• Hooks
• Page
• Application

Install

npm install dynamic-render

Getting Started

For full demo please see Demo Folder

1. First create page configuration
2. Create an application and register pages
3. Start Dynamic Render

const dynamicRender = require('dynamic-render');
const examplePage = dynamicRender.page({
    name: 'example-page',
    hooks: [],
    interceptors: [],
    matcher: '/example/:pageParam',
    followRedirects: false, // Dynamic render follow the requests redirects and respond directly to browser. Default "followRedirects" is true.
});

dynamicRender.application('example-web', {
  pages: [examplePage],
  origin: 'https://example-site.com'
});

dynamicRender
  .start()
  .then(port => {
    console.log(`Prerender listening on ${port}`);
  });

(Optional) You can pass configuration parameters for debugging purposes

const config = {
  puppeteer: {
    headless: false,
    ignoreHTTPSErrors: true,
    devtools: true,
  },
  port: 8080
}

dynamicRender
  .start(config)
  .then(port => {
    console.log(`Prerender listening on ${port}`);
  });

Now you can send request to http://localhost:8080/render/example-web/example/35235657, dynamic render will respond with rendered content.

Render Cycle

---

Interceptors

Interceptors are responsible for modifying or blocking http requests. For optimizing rendering performance you may want to mock assets. Best use case for interceptors is handling image requests.

const dynamicRender = require('dynamic-render');
const placeholderPng = fs.readFileSync(path.join(__dirname, './png_placeholder'));
const imageInterceptor = dynamicRender.interceptor({
  name: 'Image Interceptor',
  handler: (req, respond) => {
    const url = req.url();
    if (url.endsWith('png')) {
      respond({
        body: placeholderPng,
        contentType: 'image/png'
      })
    }
  }
});

Interceptors can be shared across pages. To register an interceptor to a page you use interceptors property of it.

const examplePage = dynamicRender.page({
    name: 'example-page',
    hooks: [],
    interceptors: [imageInterceptor],
    matcher: '/example/:pageParam'
});

---

Hooks

Hooks are responsible for modifying loaded dom content. Best use case for hooks is removing unnecessary assets for Google. As page is already rendered with clientside javascript, they are useless now.

const dynamicRender = require('dynamic-render');
const clearJS = dynamicRender.hook({
  name: 'Clear JS',
  handler: async (page) => {
    await page.evaluate(() => {
      const elements = document.getElementsByTagName('script');
      while (elements.length > 0) {
        const [target] = elements;
        target.parentNode.removeChild(target);
      }
    });
  },
})

Hooks can be shared across pages. To register a hook to a page you use hooks property of it.

Example hooks:

Remove comments from DOM

Comments for humans, not for search engine scrapers.

/* eslint-disable no-cond-assign */
import dynamicRender from 'dynamic-render';

const clearComments = dynamicRender.hook({
  name: 'Clear comments',
  handler: async (page) => {
    await page.evaluate(() => {
      const nodeIterator = document.createNodeIterator(
        document,
        NodeFilter.SHOW_COMMENT,
      );
      let currentNode;

      while (currentNode = nodeIterator.nextNode()) {
        currentNode.parentNode.removeChild(currentNode);
      }
    });
  },
});

Async DOM element handler

We're reducing DOM asset size with CDN's. Also we detach from DOM, when user intercept specific area, we load it. Called as lazy load. Dynamic render can trigger intercept and wait lazy loading.

import dynamicRender from 'dynamic-render';

const loader = (name, container, element, lazyImage) => dynamicRender.hook({
  name,
  handler: async (page) => {
    const containerExists = await page.evaluate((container) => {
      const containerElement = document.querySelector(container);
      if (containerElement) {
        window.scrollBy(0, containerElement.offsetTop);
      }
      return Promise.resolve(!!containerElement);
    }, container);
    if (containerExists) {
      await page.waitForSelector(`${element} ${lazyImage}`, {
        timeout: 1000,
      }).catch(() => true);
    }
  },
});

// Usage:
/*
const waitForLoad = loader('name-it', '#spesific-div', '#spesific-part', 'img[src*="cool-cdn-url"]');
*/

Feel free to publish your killer hooks with world!

Usage:

const examplePage = dynamicRender.page({
    name: 'example-page',
    hooks: [clearJS],
    interceptors: [],
    matcher: '/example/:pageParam'
});

---

Page

Pages represent your controllers. An application might have multiple pages and you can provide different configurations for them.

const productDetailPage = dynamicRender.page({
  name: 'product-detail',
  hooks: [jsAssetCleaner],
  interceptors: [imageInterceptor],
  matcher: '/example/:pageParam',
  emulateOptions: {
    userAgent: 'Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/604.1.38 (KHTML, like Gecko) Version/11.0 Mobile/15A372 Safari/604.1',
    viewport: {
      width: 414,
      height: 736,
      deviceScaleFactor: 3,
      isMobile: true,
      hasTouch: true,
      isLandscape: false
    }
  },
  waitMethod: 'load',
  query: {
    test: 12345,
    qa: 'GA-XXXXX'
  },
  followRedirects: true
});

Property	Required	Description
name	true	Name of the page
hooks	false	Array of Hooks
interceptors	false	Array of Interceptors
matcher	true	Matches url with page. Express-like matchers are accepted
emulateOptions	false	Default values are provided below, rendering options
waitMethod	false	Default value is 'load', you can check Puppeteer wait methods
query	false	Default value is '{}', you can pass query strings to matched url
followRedirects	false	Default value is 'true', you can pass false for not for follow incoming redirects.

Default emulate options are

{
    userAgent: 'Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/604.1.38 (KHTML, like Gecko) Version/11.0 Mobile/15A372 Safari/604.1',
    viewport: {
        width: 414,
        height: 736,
        deviceScaleFactor: 3,
        isMobile: true,
        hasTouch: true,
        isLandscape: false
    }
}

Page emulate options are not required when emulateOptions provided at application level. If you want to override application level configuration you can use page emulate options.

---

Application

Applications are the top level configuration for hosts

dynamicRender.application('mobile-web', {
  pages: [productDetailPage],
  origin: 'https://m.trendyol.com'
});

Property	Required	Description
pages	true	Array of Pages
origin	true	http://targethost.com
emulateOptions	false	Application level emulate options that affects all pages
plugins	false	Plugin instance can be used for custom caching strategies

Plugins

Plugins can be injected into applications to program custom caching strategies.

class CachePlugin implements Plugin {
  private cache: Map<string, RenderResult> = new Map();

  async onBeforeStart(){
    console.log('Make some connections');
  }

  async onBeforeRender(page: Page, url: string){
    const existing = this.cache.get(url);

    if(existing){
      return existing;
    }
  }

  async onAfterRender(page: Page, url: string, renderResult: RenderResult){
    this.cache.set(url, renderResult);
  }
}

dynamicRender.application('mobile-web', {
  pages: [productDetailPage],
  origin: 'https://m.trendyol.com',
  plugins: [new CachePlugin()]
});