x-ray is a web scraping library for Node.js. It can use various pluggable drivers such as HTTP or PhantomJS.
Resources:
Questions tagged [x-ray]
107 questions
11
votes
1 answer
x-ray-phantom authentication, unable to effectively login
I'm really can't find any example of using x-ray and .driver(phantom()) for authentication..
I've trawled through the documentation for x-ray and x-ray-phantom yet can't find any help.
Babra Cunningham
- 2,949
- 1
- 23
- 50
10
votes
2 answers
NodeJS x-ray web-scraper: how to follow links and get content from sub page
So I am trying to scrape some content with node.js x-ray scraping framework. While I can get the content from a single page I can't get my head around on how to follow links and get content from a subpage in one go.
There is a sample on x-ray github…
Ales Maticic
- 1,895
- 3
- 13
- 27
7
votes
1 answer
How to fix encoding in x-ray (NodeJS scraping library) response?
The following script is working perfectly in my NodeJS server, but rarely it returns response like this, when I'm trying to scrape some Cyrillic websites.
Script
x(url, {
name: 'title',
ogDescription:…
stkvtflw
- 12,092
- 26
- 78
- 155
3
votes
1 answer
Contour Axis for Image
For a radiographic scan, I have been able to acquire the contours.
I would be interested to find the center axis. How could I do it in python?
Here is my code for contours:
import cv2
img = cv2.imread("A.png")
imgray = cv2.cvtColor(img,…
Adam Merckx
- 1,122
- 1
- 14
- 31
3
votes
1 answer
Jest async test times out if "expect" called within async function. Works sometimes. "Async callback was not invoked within timeout specified"
I am using Jest with JS and trying to write a test around the X-ray JS library, a web scraping toolkit. The following is the test. This is using Jest 18.x and the latest x-ray js as of 02/20/2017.
const htmlResponse =…
3-MeO-PCE
- 31
- 1
- 2
3
votes
1 answer
How to scrape bad formatting HTML code with x-ray
I am using the x-ray module for the first time.
I have no problem for using it but I have some issue when I try to scrape data in a bad formatting HTML code.
For example, if I try to scrape this HTML code from a website:
Cyril Vandenberghe
- 31
- 1
- 5
3
votes
1 answer
NodeJS X-Ray Hide IP Address
Is it possible to change your IP Address and User Agent when using NodeJS/X-Ray to make requests to an external site?
Randy
- 71
- 1
- 8
2
votes
4 answers
AWS X-RAY [Error] Sending segment batch failed with: NoCredentialProviders: no valid providers in chain. Deprecated
I'm using X-Ray Daemon with Fargate. Unfortunately in the logs of the daemon I see errors like:
[Error] Sending segment batch failed with: NoCredentialProviders: no
valid providers in chain. Deprecated. For verbose messaging…
lilfra92
- 41
- 2
2
votes
0 answers
AWS X-Ray TraceId not found
In my AWS environment I am able to find Root X-ray TraceId in CloudWatch Logs however it is not showing in X-Ray trace Search
Able to find in CloudWatch logs using this Query
Gaurav Sharma
- 51
- 6
2
votes
1 answer
"The label is already a ground truth" happens when importing the images
When I import 100 images into the dataset, but only 99 images are imported successfully. A warning is reported:
"Failed to import some images".
The error messages: The label is already a ground truth.
The images I was importing are Chest X-Ray…
Guanfeng Chen
- 21
- 1
2
votes
0 answers
Allure2 How to integrate it with Jira Xray
I am trying to integrate Allure2 reports with Jira plugin called Xray. So far i managed to generate those reports on local machine and on Jenkins server in pipeline job with Allure plugin.
Also i see that there is an actual Allure plugin called…
anynous
- 21
- 5
2
votes
2 answers
Node x-ray crawling data from collection of url
I'm trying to scrape a list in a site that leads to other pages that has the same formatting.
I was able to create a collection of all the a tags, but when I try to visit a collection of pages, the key I try to create with it doesn't get added in my…
JoshChang
- 43
- 1
- 7
2
votes
2 answers
node.js timeout not firing
I'm using the x-ray library to scrape a specific website for some content and save it in a JSON file.
I'd like to get the status every hour, but Node does not execute my setTimeout.
This is my code:
var db, record, x;
x = require('x-ray')();
db =…
ROAL
- 1,749
- 1
- 14
- 21
2
votes
1 answer
Trying to get information out of X-ray Scraping
I'm having some troubles with the X-ray. I feel that isn't the x-ray specifically but instead the javascript.
var Xray = require('x-ray');
var xray = Xray();
var brasil = [];
function stringToNumber(str){
var number;
for(var…
Renan Cidale
- 842
- 2
- 10
- 23
2
votes
0 answers
Submit form with x-ray
I just found a neat web scraper for node called xray, is it possible to use this library to populate form data and submit it? Or is this library only for reading data from a webpage?
ThomasReggi
- 55,053
- 85
- 237
- 424