TLDR: Is it possible to leverage the vulnerability detection abilities of npm audit as a restful service instead of the current CLI implementation?
npm provides automatic vulnerability scanning on every install request against the Node Security Platform (NSP) vulnerability database and warns you if you try to use unsafe code. Furthermore, npm audit recursively analyzes your dependency trees to identify specifically what’s insecure, recommend a replacement, or fix it automatically with npm audit fix.
This functionality is great and I would like to be able to utilize this vulnerability scanning capability within a web application. So why would I want to do this?
It seems like most companies host an internal JFrog Repository, which constantly needs to updated and maintained just to mirror npmjs. However, a more efficient approach (in my mind) would be to create a simple web application with mitmproxy embedded within it. This web application would then function more like a proxy and would allow one to filter out npm requests based on custom business logic and/or npm audit vulnerability report findings. This would have the benefit of allowing one to customize their risk assessment tolerance as well as leveraging npmjs to distribute the requested libraries. As a consequence, this would drive out the need for companies to host any internal JFrog instances and could potentially lower costs by instead having npmjs deal with the hosting of said libraries.
Listed below is part of the an npm audit report:
$ npm audit
sample audit report:
=== npm audit security report ===
# ... Removed unnecessary details
# Run npm install jquery@3.4.1 to resolve 1 vulnerability
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ Moderate │ Prototype Pollution │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package │ jquery │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ jquery │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path │ jquery │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info │ https://nodesecurity.io/advisories/796 │
└───────────────┴──────────────────────────────────────────────────────────────┘
found 88 vulnerabilities (63 low, 10 moderate, 15 high) in 36801 scanned packages
run `npm audit fix` to fix 1 of them.
87 vulnerabilities require semver-major dependency updates.
I see that npm audit is leveraging the following url for vulnerability detection:
https://nodesecurity.io/advisories/<id>
where <id> is a number representing the library in question. In my example: jquery => 796.
I don't know how to replicate this component name to id mapping on my end, short of a brute force manual effort, in order to scrape the response for vulnerability details. I understand the inner workings of this API are intentionally obfuscated for security reasons and generally most security providers would like to make money for their services.
With that being said, for a first pass knowing whether the <package>@<version> is high/medium/low vulnerability would be sufficient. I see there is an embedded <script> tag within the html page containing the vulnerability details:
<script integrity="sha512-2KUTRVRvbDU3H6wROMklMMJqo9viHDRE+eOC56AIunI3PWKmCX1sVagJux/7BdYxpbbdgUi2sDJGhHEl499Tzw==">window.__context__ = {"context":{"advisoryData":{"id":796,"created":"2019-04-02T21:06:11.895Z","updated":"2019-04-23T14:29:39.788Z","deleted":null,"title":"Prototype Pollution","found_by":{"link":"","name":"asgerf"},"reported_by":{"link":"","name":"asgerf"},"module_name":"jquery","cves":["CVE-2019-5428"],"vulnerable_versions":"\u003c3.4.0","patched_versions":"\u003e=3.4.0","overview":"Versions of `jquery` prior to 3.4.0 are vulnerable to Prototype Pollution. The extend() method allows an attacker to modify the prototype for `Object` causing changes in properties that will exist on all objects.","recommendation":"Upgrade to version 3.4.0 or later.","references":"- [HackerOne Report](https://hackerone.com/reports/454365)","access":"public","severity":"moderate","cwe":"CWE-471","url":"https://npmjs.com/advisories/796","urls":{"detail":"/v1/advisories/advisory/796","prev":"/v1/advisories/advisory/795","next":"/v1/advisories/advisory/797"},"formatted":{"overview":"\u003cp\u003eVersions of \u003ccode\u003ejquery\u003c/code\u003e prior to 3.4.0 are vulnerable to Prototype Pollution. The extend() method allows an attacker to modify the prototype for \u003ccode\u003eObject\u003c/code\u003e causing changes in properties that will exist on all objects.\u003c/p\u003e\n","recommendation":"\u003cp\u003eUpgrade to version 3.4.0 or later.\u003c/p\u003e\n","references":"\u003cul\u003e\n\u003cli\u003e\u003ca href=\"https://hackerone.com/reports/454365\" rel=\"nofollow\"\u003eHackerOne Report\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n","created":"Apr 2nd, 9:06:11 pm","updated":"Apr 23rd, 2:29:39 pm"}},"events":[{"id":1419,"advisory_id":796,"created":"2019-04-23T14:29:39.821Z","type":"published","message":"Advisory Published","formatted":{"created":"Apr 23rd, 2019"}},{"id":1354,"advisory_id":796,"created":"2019-04-02T21:06:11.904Z","type":"reported","message":"Reported by asgerf","formatted":{"created":"Apr 2nd, 2019"}}],"user":{"tfa":false,"name":"wright1242","isStaff":false,"deactivated":null,"avatars":{"small":"https://s.gravatar.com/avatar/c273b03158ed9f9f045f476897b235fa?size=50\u0026default=retro","medium":"https://s.gravatar.com/avatar/c273b03158ed9f9f045f476897b235fa?size=100\u0026default=retro","large":"https://s.gravatar.com/avatar/c273b03158ed9f9f045f476897b235fa?size=496\u0026default=retro"},"resource":{"fullname":"Nathan Wright"},"is_delegated":false,"email_verified":true,"created":{"ts":1553877333697,"rel":"3 months ago"},"updated":"2019-03-29T16:35:33.695Z"},"csrftoken":"plyisfZBsPE1Ede0ico2RTod6B60nd7l1qHPvREr-mw","notifications":[],"npmExpansions":["Nimble Porridge Muncher","Now, Push Me","Next Phenomenal Microbrewery","npm promulgates marsupials","Newton's Poleless Magnet","Nested Public Modules","New Powerful Machines","No Prize Money","Nostalgic Pickled Mango","Neolithic Populous Metropolis"],"isNpme":false},"chunks":{"commons":["commons.4d94cbb36d7d9f02c2f4.js","commons.4d94cbb36d7d9f02c2f4.js.map"],"styles":["styles.a34b113ba89c0a069aa9.css","minicssextractbug.ece81719b14e4fb51acb.js","styles.a34b113ba89c0a069aa9.css.map","minicssextractbug.ece81719b14e4fb51acb.js.map"],"advisories/detail":["advisories/detail.e640a34c8a03ac2c51e8.js","advisories/detail.e640a34c8a03ac2c51e8.js.map"],"advisories/list":["advisories/list.49eaf09da2c66523e9cd.js","advisories/list.49eaf09da2c66523e9cd.js.map"],"advisories/report":["advisories/report.6b5a536458e618cb3ae5.js","advisories/report.6b5a536458e618cb3ae5.js.map"],"advisories/versions":["advisories/versions.5ce20fcc3f3f3e620292.js","advisories/versions.5ce20fcc3f3f3e620292.js.map"],"auth/cli":["auth/cli.31c9a365866841fcc4fe.js","auth/cli.31c9a365866841fcc4fe.js.map"],"auth/common-passwords":["auth/common-passwords.32150bf4a63195186b9e.js","auth/common-passwords.32150bf4a63195186b9e.js.map"],"auth/escalate":["auth/escalate.5d3004f00377e61c65ff.js","auth/escalate.5d3004f00377e61c65ff.js.map"],"auth/forgot":["auth/forgot.4b4e93ca04ea1741a235.js","auth/forgot.4b4e93ca04ea1741a235.js.map"],"auth/forgot-sent":["auth/forgot-sent.b6a321ab13288fbdb321.js","auth/forgot-sent.b6a321ab13288fbdb321.js.map"],"auth/invite-signup":["auth/invite-signup.1374d727ca7f216f4df6.js","auth/invite-signup.1374d727ca7f216f4df6.js.map"],"auth/login":["auth/login.8d9b5fe8a19cbc186849.js","auth/login.8d9b5fe8a19cbc186849.js.map"],"auth/otp":["auth/otp.17f71c286c4ef5838c10.js","auth/otp.17f71c286c4ef5838c10.js.map"],"auth/reset-password":["auth/reset-password.b20251afbd7655b491c1.js","auth/reset-password.b20251afbd7655b491c1.js.map"],"auth/signup":["auth/signup.38e2de18a3d7d49e1901.js","auth/signup.38e2de18a3d7d49e1901.js.map"],"auth/sso-signup":["auth/sso-signup.8c3feddbe01f02ad701b.js","auth/sso-signup.8c3feddbe01f02ad701b.js.map"],"billing/detail":["billing/detail.7c30c25000cb18635fad.js","billing/detail.7c30c25000cb18635fad.js.map"],"billing/downgrade":["billing/downgrade.3be55cca5333d74807d5.js","billing/downgrade.3be55cca5333d74807d5.js.map"],"billing/upgrade":["billing/upgrade.aa74d23e03f79c2b1e56.js","billing/upgrade.aa74d23e03f79c2b1e56.js.map"],"contact/contact":["contact/contact.8662906bb6004554b3f2.js","contact/contact.8662906bb6004554b3f2.js.map"],"debug/badstatus":["debug/badstatus.c7bb04c58ae395906dbf.js","debug/badstatus.c7bb04c58ae395906dbf.js.map"],"debug/detail":["debug/detail.34b54844c3ec9f69e471.js","debug/detail.34b54844c3ec9f69e471.js.map"],"debug/failcomponent":["debug/failcomponent.d1f8803e2009818ef71d.js","debug/failcomponent.d1f8803e2009818ef71d.js.map"],"egg/egg":["egg/egg.4e4902966f9314b37154.js","egg/egg.4e4902966f9314b37154.js.map"],"enterprise/complete":["enterprise/complete.89eaa305053a6c0f8989.js","enterprise/complete.89eaa305053a6c0f8989.js.map"],"enterprise/license-paid":["enterprise/license-paid.ebe1bfa16a3d49069d9b.js","enterprise/license-paid.ebe1bfa16a3d49069d9b.js.map"],"enterprise/license-purchase":["enterprise/license-purchase.33b546c059bd99696cf0.js","enterprise/license-purchase.33b546c059bd99696cf0.js.map"],"enterprise/on-site-buy-now":["enterprise/on-site-buy-now.537497d98021ab3a5cb2.js","enterprise/on-site-buy-now.537497d98021ab3a5cb2.js.map"],"enterprise/on-site-contact-confirmation":["enterprise/on-site-contact-confirmation.30cef6ea069bb5b8d238.js","enterprise/on-site-contact-confirmation.30cef6ea069bb5b8d238.js.map"],"enterprise/on-site-trial":["enterprise/on-site-trial.c4299d1451374df8d233.js","enterprise/on-site-trial.c4299d1451374df8d233.js.map"],"enterprise/orgs-terms":["enterprise/orgs-terms.1d97d471a2406c7a0bfa.js","enterprise/orgs-terms.1d97d471a2406c7a0bfa.js.map"],"enterprise/signup-confirmation":["enterprise/signup-confirmation.01182145a57b51bf81d6.js","enterprise/signup-confirmation.01182145a57b51bf81d6.js.map"],"errors/not-found":["errors/not-found.233c66ddecbbf24ac4fc.js","errors/not-found.233c66ddecbbf24ac4fc.js.map"],"errors/server":["errors/server.dd29f86cfe1f6df5386d.js","errors/server.dd29f86cfe1f6df5386d.js.map"],"errors/template":["errors/template.66c3e6b9be4cdeeb1b31.js","errors/template.66c3e6b9be4cdeeb1b31.js.map"],"flatpage/flatpage":["flatpage/flatpage.a15b631354b26980e9d2.js","flatpage/flatpage.a15b631354b26980e9d2.js.map"],"homepage/homepage":["homepage/homepage.110dbed7fffb8ed42685.js","homepage/homepage.110dbed7fffb8ed42685.js.map"],"homepage/homepage-logged-in":["homepage/homepage-logged-in.8797a9ea2201ab4336cc.js","homepage/homepage-logged-in.8797a9ea2201ab4336cc.js.map"],"npme-2/invite":["npme-2/invite.2db2f811ab5b4ca10d5f.js","npme-2/invite.2db2f811ab5b4ca10d5f.js.map"],"npme-2/invites":["npme-2/invites.8878a423875defcf6c76.js","npme-2/invites.8878a423875defcf6c76.js.map"],"npme-2/login":["npme-2/login.a9b772e49ae0412bf666.js","npme-2/login.a9b772e49ae0412bf666.js.map"],"npme-2/overrides/components/tutorials/creating-org":["npme-2/overrides/components/tutorials/creating-org.71f7d5901781c193fb73.js","npme-2/overrides/components/tutorials/creating-org.71f7d5901781c193fb73.js.map"],"npme-2/overrides/components/tutorials/default-registry":["npme-2/overrides/components/tutorials/default-registry.475fb9f18556bd682949.js","npme-2/overrides/components/tutorials/default-registry.475fb9f18556bd682949.js.map"],"npme-2/overrides/components/tutorials/installing-package":["npme-2/overrides/components/tutorials/installing-package.2b04ab2356a096fd5a49.js","npme-2/overrides/components/tutorials/installing-package.2b04ab2356a096fd5a49.js.map"],"npme-2/overrides/components/tutorials/publishing-package":["npme-2/overrides/components/tutorials/publishing-package.5697dc8704d72ac9ced0.js","npme-2/overrides/components/tutorials/publishing-package.5697dc8704d72ac9ced0.js.map"],"npme-2/overrides/components/tutorials/tabs":["npme-2/overrides/components/tutorials/tabs.df1dac1eefb18c4859bc.js","npme-2/overrides/components/tutorials/tabs.df1dac1eefb18c4859bc.js.map"],"npme-2/overrides/homepage":["npme-2/overrides/homepage.c385707ae0df1411a2a1.js","npme-2/overrides/homepage.c385707ae0df1411a2a1.js.map"],"npme-2/overrides/orgs/create":["npme-2/overrides/orgs/create.796a1ec58f26a83b0b55.js","npme-2/overrides/orgs/create.796a1ec58f26a83b0b55.js.map"],"npme-2/reports":["npme-2/reports.c848851d64a15951a1ef.js","npme-2/reports.c848851d64a15951a1ef.js.map"],"npme-2/settings":["npme-2/settings.d0c798bb186ce82f3ea8.js","npme-2/settings.d0c798bb186ce82f3ea8.js.map"],"npme-2/setup":["npme-2/setup.8ebd15e786c914775153.js","npme-2/setup.8ebd15e786c914775153.js.map"],"npme-2/sso-config":["npme-2/sso-config.28c27038f210cf50638a.js","npme-2/sso-config.28c27038f210cf50638a.js.map"],"npme-2/users":["npme-2/users.eeb8e1c64514e5683330.js","npme-2/users.eeb8e1c64514e5683330.js.map"],"npme/invite":["npme/invite.ffae73172929956e34eb.js","npme/invite.ffae73172929956e34eb.js.map"],"npme/invites":["npme/invites.44e371e54eac0a082e0d.js","npme/invites.44e371e54eac0a082e0d.js.map"],"npme/login":["npme/login.ab849e614586290dfb37.js","npme/login.ab849e614586290dfb37.js.map"],"npme/overrides/components/tutorials/creating-org":["npme/overrides/components/tutorials/creating-org.471161dde8734e77baa4.js","npme/overrides/components/tutorials/creating-org.471161dde8734e77baa4.js.map"],"npme/overrides/components/tutorials/default-registry":["npme/overrides/components/tutorials/default-registry.17619197ac9ebf75f78b.js","npme/overrides/components/tutorials/default-registry.17619197ac9ebf75f78b.js.map"],"npme/overrides/components/tutorials/installing-package":["npme/overrides/components/tutorials/installing-package.e55b3915848c09265955.js","npme/overrides/components/tutorials/installing-package.e55b3915848c09265955.js.map"],"npme/overrides/components/tutorials/publishing-package":["npme/overrides/components/tutorials/publishing-package.0e248bac8e84760c3a3c.js","npme/overrides/components/tutorials/publishing-package.0e248bac8e84760c3a3c.js.map"],"npme/overrides/components/tutorials/tabs":["npme/overrides/components/tutorials/tabs.1f4c0ff4d1338c5cb611.js","npme/overrides/components/tutorials/tabs.1f4c0ff4d1338c5cb611.js.map"],"npme/overrides/homepage":["npme/overrides/homepage.4955c4963ed9fa476105.js","npme/overrides/homepage.4955c4963ed9fa476105.js.map"],"npme/overrides/orgs/create":["npme/overrides/orgs/create.a7fa242e75db505a14cc.js","npme/overrides/orgs/create.a7fa242e75db505a14cc.js.map"],"npme/settings":["npme/settings.58e57b118bbd7878e2ed.js","npme/settings.58e57b118bbd7878e2ed.js.map"],"npme/setup":["npme/setup.c348ba66cd10e64fba12.js","npme/setup.c348ba66cd10e64fba12.js.map"],"npme/sso-config":["npme/sso-config.fc863259ffafbadaac78.js","npme/sso-config.fc863259ffafbadaac78.js.map"],"npme/users":["npme/users.7bf881838868fa2e8146.js","npme/users.7bf881838868fa2e8146.js.map"],"orgs/create":["orgs/create.25acfbc854056d91faab.js","orgs/create.25acfbc854056d91faab.js.map"],"orgs/detail":["orgs/detail.a0fc09a1dafcc608f604.js","orgs/detail.a0fc09a1dafcc608f604.js.map"],"orgs/invite":["orgs/invite.bb4dd212f2a1638cd111.js","orgs/invite.bb4dd212f2a1638cd111.js.map"],"orgs/upgrade":["orgs/upgrade.720c3caf2998a6f28cd5.js","orgs/upgrade.720c3caf2998a6f28cd5.js.map"],"package-list/dependents-list":["package-list/dependents-list.2dce05752934fcbcbe4f.js","package-list/dependents-list.2dce05752934fcbcbe4f.js.map"],"package-list/most-depended":["package-list/most-depended.f7001bbcaa0641bf4f40.js","package-list/most-depended.f7001bbcaa0641bf4f40.js.map"],"package-list/recently-updated":["package-list/recently-updated.37fbdf8fec827ddacad3.js","package-list/recently-updated.37fbdf8fec827ddacad3.js.map"],"package/package":["package/package.a8b3c84300ae1382adf1.js","package/package.a8b3c84300ae1382adf1.js.map"],"partners/detail":["partners/detail.455f79e0b6e62b2b62c8.js","partners/detail.455f79e0b6e62b2b62c8.js.map"],"partners/join":["partners/join.64518905a4506bd65cb0.js","partners/join.64518905a4506bd65cb0.js.map"],"partners/thanks":["partners/thanks.ff01ac06bf6bc9dca957.js","partners/thanks.ff01ac06bf6bc9dca957.js.map"],"profile/profile":["profile/profile.225a06aa9545bb306666.js","profile/profile.225a06aa9545bb306666.js.map"],"search/search":["search/search.7a1e54cf3d045148dbc1.js","search/search.7a1e54cf3d045148dbc1.js.map"],"settings/change-password":["settings/change-password.70b297e43c57d042ce46.js","settings/change-password.70b297e43c57d042ce46.js.map"],"settings/email":["settings/email.fadfc72c579bb081162b.js","settings/email.fadfc72c579bb081162b.js.map"],"settings/memberships":["settings/memberships.410836fc226dd288ffb8.js","settings/memberships.410836fc226dd288ffb8.js.map"],"settings/packages":["settings/packages.64c297193727a8e51542.js","settings/packages.64c297193727a8e51542.js.map"],"settings/profile":["settings/profile.833a150cc274224f5f70.js","settings/profile.833a150cc274224f5f70.js.map"],"teams/create":["teams/create.b9f70a2ae9de1915d420.js","teams/create.b9f70a2ae9de1915d420.js.map"],"teams/detail":["teams/detail.d4664bced272f9d0a3ad.js","teams/detail.d4664bced272f9d0a3ad.js.map"],"teams/list":["teams/list.261cf5f94b9192a3daab.js","teams/list.261cf5f94b9192a3daab.js.map"],"teams/packages":["teams/packages.71096da465263d66286d.js","teams/packages.71096da465263d66286d.js.map"],"teams/users":["teams/users.674a8b5b623059964462.js","teams/users.674a8b5b623059964462.js.map"],"tfa/enable":["tfa/enable.18fb4a852c69318b5589.js","tfa/enable.18fb4a852c69318b5589.js.map"],"tfa/showTFAQRCode":["tfa/showTFAQRCode.eb7002f5007a27821807.js","tfa/showTFAQRCode.eb7002f5007a27821807.js.map"],"tfa/showTFASuccess":["tfa/showTFASuccess.d060dc3061cf308203c2.js","tfa/showTFASuccess.d060dc3061cf308203c2.js.map"],"tfa/tfa-mode-selection":["tfa/tfa-mode-selection.a16db0ac7e3f02c6f5d3.js","tfa/tfa-mode-selection.a16db0ac7e3f02c6f5d3.js.map"],"tfa/tfa-password-entry":["tfa/tfa-password-entry.be0ca4c5433e4802b5ac.js","tfa/tfa-password-entry.be0ca4c5433e4802b5ac.js.map"],"tokens/create":["tokens/create.0a64e73c9a20dc823d2a.js","tokens/create.0a64e73c9a20dc823d2a.js.map"],"tokens/list":["tokens/list.1d67b6a2423c57d4efcd.js","tokens/list.1d67b6a2423c57d4efcd.js.map"],"vouchers/view":["vouchers/view.cc023324f08f48ef3082.js","vouchers/view.cc023324f08f48ef3082.js.map"]},"hash":"4d94cbb36d7d9f02c2f4","name":"advisories/detail","containerId":"app","headerName":"x-spiferack","publicPath":"https://static.npmjs.com/"}</script>
I am open to any other open source security scanning tools, such as: Snyk, OWASP, etc... as long as I can leverage this vulnerability detection abilities as a web service. Any ideas on what else to try/use?
Any help would be greatly appreciated!
Update:
It appears as though Node Security is leveraging the National Vulnerability Database (NVD) for open source vulnerabilities and have mapped modules to Common Vulnerabilities and Exposures (CVEs). One can acquire the entire CVE dataset in multiple forms here. Perhaps this data could be reversed mapped? I see within the embedded <script> tag there are numerous fields for the module in question. The two fields of immediate relevance are: cves, and module_name. Where module_name points to the module in question, in my example jquery, and cves appears to be a one to one mapping from the CVE dataset mentioned above. This would allow one to read the entire dataset into a database and use this database as a source of truth for lookups. So the question really becomes:
How is Node Security mapping CVEs to module_names? Is this a manual effort or are there more columns/fields in an alternate dataset?
Update 2
Both NVD and Snyk, offer RSS feeds for vulnerability detection of libraries. Under the hood, this is exactly what npm audit is using to determine high/med/low vulnerabilities when you install a library or run an audit. These RSS feeds come in multiple formats and are actually easy to parse. Additionally, they have the mappings of modules to vulnerabilities baked in.
With that being said, if you want to leverage these open source scanners, one must abide by their designated licenses and rules for usage. For example Snyx's rules for usage are the following for their RSS feed:
Snyk's Vulnerability DB RSS feed. This DB (feed and repository) is licensed under the AGPL-v3 license, which often allows use internally, but prohibits embedding the DB in another product or service, unless that product and provided service are open source and under the AGPL-v3 license.** For a different license to Snyk's vulnerability DB, please contact us at contact@snyk.io**
As for some of the concerns noted:
You would be leveraging npmjs (which is open sourced) and that's the point. Npmjs would hold all your libraries, and you would be using npmjs as it was intended. You would pay npmjs directly if you needed more privacy, such as with privately scoped modules for internal use. Sorry if I made this unclear, as the purpose was to only use npmjs and not pay a third party entity to host a JFrog repo for you.
As for the concern about licenses, you should always abide by the laws governing the use or redistribution of software.
