puppeteer-page-proxy
Advanced tools
puppeteer-page-proxy - npm Package Compare versions
Comparing version 1.2.3 to 1.2.4
| # Change log | ||
| ### [1.2.4] - 2020-05-18 | ||
| #### Changes | ||
| - Fix 'net::ERR_FAILED' by updating package to work with latest Got ([#16](https://github.com/Cuadrix/puppeteer-page-proxy/issues/16), [#14](https://github.com/Cuadrix/puppeteer-page-proxy/issues/14)) | ||
| - Added an explanation addressing site insecurity ([#9](https://github.com/Cuadrix/puppeteer-page-proxy/issues/9), [#12](https://github.com/Cuadrix/puppeteer-page-proxy/issues/12)) | ||
| - Removed type enforcing | ||
| ### [1.2.3] - 2020-02-14 | ||
@@ -3,0 +8,0 @@ #### Changes |
| { | ||
| "name": "puppeteer-page-proxy", | ||
| "description": "Additional Node.js module to use with 'puppeteer' for setting proxies per page basis.", | ||
| "version": "1.2.3", | ||
| "version": "1.2.4", | ||
| "author": "Cuadrix <cuadrix12000@gmail.com> (https://github.com/Cuadrix)", | ||
@@ -25,9 +25,8 @@ "homepage": "https://github.com/Cuadrix/puppeteer-page-proxy", | ||
| "dependencies": { | ||
| "got": "^10.5.5", | ||
| "got": "^11.1.4", | ||
| "http-proxy-agent": "^4.0.1", | ||
| "https-proxy-agent": "^5.0.0", | ||
| "socks-proxy-agent": "^5.0.0", | ||
| "tough-cookie": "^3.0.1", | ||
| "type-dragoon": "^1.0.0" | ||
| "tough-cookie": "^4.0.0" | ||
| } | ||
| } |
@@ -1,2 +0,2 @@ | ||
| # puppeteer-page-proxy <img src="https://i.ibb.co/kQrN9QJ/puppeteer-page-proxy-logo.png" align="right" width=150 height=150/> | ||
| # puppeteer-page-proxy <img src="https://i.ibb.co/kQrN9QJ/puppeteer-page-proxy-logo.png" align="right" width="150" height="150"> | ||
| Additional Node.js module to use with **[puppeteer](https://www.npmjs.com/package/puppeteer)** for setting proxies per page basis. | ||
@@ -78,3 +78,3 @@ | ||
| page.on('request', req => { | ||
| useProxy(req, proxy); // 'req' as argument | ||
| useProxy(req, proxy); | ||
| }); | ||
@@ -84,6 +84,6 @@ await page.goto(site); | ||
| ``` | ||
| When changing proxies this way, the request object itself is passed as the first argument. Now 'proxy' can be changed every request. | ||
| Leaving it as is will have the same effect as `useProxy(page, proxy)`, meaning that the same proxy will be used for all requests within the page. | ||
| The request object itself is passed as the first argument. The proxy can now be changed every request. | ||
| Leaving it as is will have the same effect as applying a proxy for the whole page by passing in the page object as an argument. Basically, the same proxy will be used for all requests within the page. | ||
| Using it in other request listeners is also straight forward: | ||
| Using it with other interception methods is straight forward aswell: | ||
| ```js | ||
@@ -99,3 +99,3 @@ await page.setRequestInterception(true); | ||
| ``` | ||
| Since all requests can be handled exactly once, it's not possible to call other interception methods (e.g. [request.abort](https://github.com/puppeteer/puppeteer/blob/master/docs/api.md#requestaborterrorcode), [request.continue](https://github.com/puppeteer/puppeteer/blob/master/docs/api.md#requestcontinueoverrides)) after calling `useProxy`, without getting a *'Request is already handled!'* error message. This is because `puppeteer-page-proxy` internally calls [request.respond](https://github.com/puppeteer/puppeteer/blob/master/docs/api.md#requestrespondresponse) which fulfills the request. | ||
| All requests can be handled exactly once, so it's not possible to intercept the same request after a proxy has been applied to it. This means that it will not be possible to call (e.g. [request.abort](https://github.com/puppeteer/puppeteer/blob/master/docs/api.md#requestaborterrorcode), [request.continue](https://github.com/puppeteer/puppeteer/blob/master/docs/api.md#requestcontinueoverrides)) on the same request without getting a *'Request is already handled!'* error message. This is because `puppeteer-page-proxy` internally calls [request.respond](https://github.com/puppeteer/puppeteer/blob/master/docs/api.md#requestrespondresponse) which fulfills the request. | ||
@@ -122,3 +122,3 @@ **NOTE:** It is necessary to set [page.setRequestInterception](https://github.com/puppeteer/puppeteer/blob/master/docs/api.md#pagesetrequestinterceptionvalue) to true when setting proxies this way, otherwise the function will fail. | ||
| /**1*/ | ||
| // 1 | ||
| const page1 = await browser.newPage(); | ||
@@ -130,3 +130,3 @@ await useProxy(page1, proxy1); | ||
| /**2*/ | ||
| // 2 | ||
| const page2 = await browser.newPage(); | ||
@@ -141,2 +141,26 @@ await useProxy(page2, proxy2); | ||
| ## FAQ | ||
| #### How does puppeteer-page-proxy work? | ||
| It takes over the task of requesting resources from the browser to instead do it internally. This means that the requests that the browser is usually supposed to make directly, are instead intercepted and made indirectly via Node using a requests library. This naturally means that Node also receives the responses that the browser would have normally received from those requests. For changing the proxy, the requests are routed through the specified proxy server using ***-proxy-agent**'s. The responses are then forwarded back to the browser as mock/simulated responses using the [request.respond](https://github.com/puppeteer/puppeteer/blob/master/docs/api.md#requestrespondresponse) method, making the browser think that a response has been received from the server, thus fulfilling the request and rendering any content from the response onto the screen. | ||
| #### Why does the browser show _"Your connection to this site is not secure"_ when connecting to **https** sites? | ||
| This is simply because the server and the browser are unable perform the secure handshakes for the connections due to the requests being intercepted and effectively blocked by Node when forwarding responses to the browser. However, despite the browser alerting of an insecure connection, the requests are infact made securely through Node as seen from the connection property of the response object: | ||
| ``` | ||
| connection: TLSSocket { | ||
| _tlsOptions: { | ||
| secureContext: [SecureContext], | ||
| requestCert: true, | ||
| rejectUnauthorized: true, | ||
| }, | ||
| _secureEstablished: true, | ||
| authorized: true, | ||
| encrypted: true, | ||
| } | ||
| ``` | ||
| While a proxy is applied, the browser is just an empty drawing board used for rendering content on the screen. All the network requests and responses, both secure and non-secure, are made by Node. Because of this, it makes no difference whether the site in the browser is shown as insecure or not. | ||
| ## Dependencies | ||
@@ -143,0 +167,0 @@ - [Got](https://github.com/sindresorhus/got) |
@@ -1,7 +0,2 @@ | ||
| const enforceTypes = require("type-dragoon"); | ||
| const lookup = async (page, lookupService = "https://api.ipify.org?format=json", isJSON = true, timeout = 30000) => { | ||
| /**/ | ||
| enforceTypes({object: page}, {string: lookupService}, {boolean: isJSON}, {number: timeout}); | ||
| /**/ | ||
| const XMLHttpRequest = async () => { | ||
@@ -8,0 +3,0 @@ return await page.evaluate((lookupService, timeout, isJSON) => { |
@@ -1,2 +0,2 @@ | ||
| module.exports = require("./core/page-proxy"); | ||
| module.exports = require("./core/proxy"); | ||
| module.exports.lookup = require("./core/lookup"); |
@@ -12,3 +12,3 @@ const WebSocket = require("ws"); | ||
| await new Promise(resolve => ws.once("open", resolve)); | ||
| /* Attach to target then get cookies */ | ||
| // Attach to target then get cookies | ||
| const sessionId = await Target.attachToTarget(ws, targetId); | ||
@@ -22,3 +22,3 @@ return await Network.getCookies(ws, sessionId); | ||
| return CookieJar.deserializeSync({ | ||
| version: 'tough-cookie@3.0.1', | ||
| version: 'tough-cookie@4.0.0', | ||
| storeType: 'MemoryCookieStore', | ||
@@ -25,0 +25,0 @@ rejectPublicSuffixes: true, |
@@ -26,8 +26,11 @@ const got = require("got"); | ||
| if (proxy.startsWith("socks")) { | ||
| return new SocksProxyAgent(proxy); | ||
| } | ||
| else if (url.startsWith("https")) { | ||
| return new HttpsProxyAgent(proxy); | ||
| return { | ||
| http: new SocksProxyAgent(proxy), | ||
| https: new SocksProxyAgent(proxy) | ||
| } | ||
| } else { | ||
| return new HttpProxyAgent(proxy); | ||
| return { | ||
| http: new HttpProxyAgent(proxy), | ||
| https: new HttpsProxyAgent(proxy) | ||
| } | ||
| } | ||
@@ -41,3 +44,3 @@ }, | ||
| headers: res.headers, | ||
| body: res.body, | ||
| body: res.body | ||
| }; | ||
@@ -44,0 +47,0 @@ } catch(error) { |
New alerts
License Policy Violation
LicenseThis package is not allowed per your license policy. Review the package's license to ensure compliance.
Found 1 instance in 1 package
Fixed alerts
License Policy Violation
LicenseThis package is not allowed per your license policy. Review the package's license to ensure compliance.
Found 1 instance in 1 package
Improved metrics
- Total package byte prevSize
- increased by13.12%
20032
- Dependency count
- decreased by-16.67%
5
- Number of lines in readme file
- increased by16.9%
166
Worsened metrics
- Lines of code
- decreased by-1.19%
250
Dependency changes
+ Added@sindresorhus/is@4.6.0(transitive)
+ Addedcacheable-lookup@5.0.4(transitive)
+ Addeddecompress-response@6.0.0(transitive)
+ Addedgot@11.8.6(transitive)
+ Addedhttp2-wrapper@1.0.3(transitive)
+ Addedmimic-response@3.1.0(transitive)
+ Addedquerystringify@2.2.0(transitive)
+ Addedquick-lru@5.1.1(transitive)
+ Addedrequires-port@1.0.0(transitive)
+ Addedresolve-alpn@1.2.1(transitive)
+ Addedtough-cookie@4.1.4(transitive)
+ Addeduniversalify@0.2.0(transitive)
+ Addedurl-parse@1.5.10(transitive)
- Removedtype-dragoon@^1.0.0
- Removed@sindresorhus/is@2.1.1(transitive)
- Removedcacheable-lookup@2.0.1(transitive)
- Removeddecompress-response@5.0.0(transitive)
- Removedduplexer3@0.1.5(transitive)
- Removedgot@10.7.0(transitive)
- Removedip-regex@2.1.0(transitive)
- Removedmimic-response@2.1.0(transitive)
- Removedp-event@4.2.0(transitive)
- Removedp-finally@1.0.0(transitive)
- Removedp-timeout@3.2.0(transitive)
- Removedto-readable-stream@2.1.0(transitive)
- Removedtough-cookie@3.0.1(transitive)
- Removedtype-dragoon@1.0.5(transitive)
- Removedtype-fest@0.10.0(transitive)
Updatedgot@^11.1.4
Updatedtough-cookie@^4.0.0