@grodno-city/alis-web-request

alis-web-request

Request data from ALIS WEB system

Installing

npm

npm install @grodno-city/alis-web-request --save-dev

yarn

yarn add @grodno-city/alis-web-request

API methods

• sendInitialQuery - make initial http query to alis library for get main page and cookie id.
• getPage - get page body from the specified url.
• getNumberedPageUrls - get firs ten numbered urls from main page.
• getNextPageUrl - get next page url from the specified url.
• processItems - to managements another functions getPage, getNextPageUrl, pushItemsToStream.
• getItems - get id with title all books on the page.
• run - run processItems function over queue with options.
• ReadableStreamItems - readable stream for read items.

Example

import Stream from 'stream';
import { sendInitialQuery, getNumberedPageUrls, run, processItems, parsePage, ReadableStreamItems } from '@grodno-city/alis-web-request';

const WritableStreamItems = new Stream.Writable({ objectMode: true });

ReadableStreamItems.pipe(WritableStreamItems);

const initParams = {
  year: 2017,
  alisEndpoint: 'http://86.57.174.45',
};

sendInitialQuery(initParams, (err, res) => {
  if (err) {
    return new Error(err);
  }

  const options = {
    alisEndpoint: initParams.alisEndpoint,
    jar: res.jar,
  };
  const $ = parsePage(res.page);
  const firstNumberedPageUrls = getNumberedPageUrls($);

  run(processItems, firstNumberedPageUrls, options);
});

const items = [];

WritableStreamItems._write = (item, encoding, done) => {
  items.push(item);
  done();
};

Authors

• Vadim Demyanchik - GitHub
• Pavel Zubkou - GitHub

License

MIT License