3

Preliminary context sharing

I am asked to manually perform a very repetitive action on a website that I do not own and for which I do not have any API access.

The only hope I have to automate these actions is to write some JavaScript and execute it on the browser just to automate the actions that I would be doing manually otherwise.

Please sorry in advance if this question already has an answer somewhere else, I'm a backend developer and in my limited knowledge of front-end I didn't manage to find any equivalence.

Explanation of the issue

Say I have to post several entries, one by one, into a form. I have written the following code (over simpified just for demonstration purposes):

//This array of Json objects is produced by an upstream service
var inputs = [
    {
       ...
    },
    {
       ...
    },
    {
       ...
    }
]

for (i = 0; i < inputs.length; i++) {
    fillSomeForms(inputs[i])
    clickSubmit() //<-- this will make the page reload, and so the script execution stop
}

The problem that I have here is very basic: after the first for iteration, when I invoke clickSubmit(), the page reloads (because the submission is a POST followed by a redirect to a "submit next" page) and so the JS stops executing.

I have tried to look around on the web for similar issues, and I've seen people tweaking the localStorage in order to resume the execution of their script. However, that seems to assume the script being a resource of the front-end code, which is not the case for me (I don't own the code, I simply inject this JS into the browser's developer console and execute it to save some time).

Is there any way to reach this purpose? I am not necessarily looking for a clean solution, just for something that could get this work and spare us some monkey work (nothing of what I'm doing here is clean, but the system administrators do not want to provide access to the REST APIs that the platform actually provide to do so).

Matteo NNZ
  • 11,930
  • 12
  • 52
  • 89
  • 2
    Another solution would be to emulate the whole process by using the Fetch API instead of actually filling the form and sending it, but without more context it's hard to say if this is actually doable (ex: depending on the CSRF implementation) – BJRINT Feb 16 '23 at 10:50
  • You might be able to adapt the idea used here https://stackoverflow.com/questions/5684303/javascript-window-open-pass-values-using-post to dynamically create (iteratively) versions of the form, fill its values, target it to a new window, submit by js from the parent page, remove the form, close the window, and repeat as necessary. – Dave Pritlove Feb 16 '23 at 13:06

2 Answers2

1

When you inject into the console, load a copy of the page into an iframe, and submit your forms from that copy:

const inputs = [ /* a convenient inputs array */ ];
const pageCopy = document.body.appendChild( document.createElement( "iframe" ) );
pageCopy.addEventListener( "load", () => {
    //The page copy has finished loading / reloading, let's submit more stuff
    if( inputs.length > 0 ) {
        const moreInput = inputs.pop();
        console.log( "Submitting inputs: ", moreInput );
        //this shouldn't work, but let's clone the current DOM into the iframe...
        pageCopy.contentDocument.body.parentElement.innerHTML =
            document.body.parentElement.innerHTML;
        fillSomeFormsInPageCopy( pageCopy.contentDocument, moreInput );
        pageCopy.contentDocument.querySelector( "#submitButtonId" ).click();
        console.log( "Clicked submit. Will wait for iframe to finish reloading..." );
        //Okay, we clicked and the iframe is reloading. This event will fire again as soon as it's done reloading, ready to submit more form data
    }
    else if( inputs.length === 0 ) {
        console.log( "Finished submitting all the inputs in the array!" );
    }
} );
pageCopy.src = document.location.href;

Please understand I can't test this code. (I'm not even sure the click() event can be fired across an iframe boundary, for security, but I hope it can.)
Hopefully you can understand how to use the pageCopy's document to find your form elements and set their values. E.g., you can use

pageCopy.contentDocument.getElementById( "form-entry-id-1" ).value =
    moreInput[ "form-entry-id-1" ];
Michael G
  • 458
  • 2
  • 9
  • I think this is a good way to go. I'm trying your proposal right now, so I've modified my function "fillSomeFormsInPageCopy" to retrieve the elements from a variable X instead of using the keyword document, and this variable X I'm passing it as "pageCopy.contentDocument" when I call the function. However, it fails to find the very first element that should normally be in the DOM of the page. What am I missing? – Matteo NNZ Feb 16 '23 at 11:24
  • Ok I found the issue. When I do pageCopy.src = location.toString(), the location.toString() returns a .php URL. I don't know what happens (I'm really bad with JS) but it tells me "Failed to load resource: the server responded with a status of 404 (Not Found)", I guess so the src of the page is empty and that's why it doesn't work. Any other way to copy directly the DOM without giving it an URL? – Matteo NNZ Feb 16 '23 at 11:27
  • No, it has to be a url; getting it to work is just tricky. I changed the code. Try changing that line to use document.location.href instead, like the example now shows. – Michael G Feb 16 '23 at 11:33
  • Same issue. If I open the contentDocument in the debugger, I can see that I have an iframe with the src that I set, and that this has a document. But the body of this document is something else completely (it's just an icon of a chat from the platform I'm using). What can this be? Any idea? – Matteo NNZ Feb 16 '23 at 11:39
  • Maybe I'm about to same something that doesn't make sense but... what about making a copy of the main HTML document and putting it into the iFrame? Makes sense or am I saying non-sense? – Matteo NNZ Feb 16 '23 at 11:42
  • @MatteoNNZ the problem is since the submit button is most likely using action on the page URL, the iframe's URL will need to be correct. I can't imagine why it would fail to load. I have to leave for work... And I can't help much more without seeing what you're seeing... so here's my last suggestion. Try this: Set the pageCopy's src, wait for the pageCopy to load, then try cloning the current page with `pageCopy.contentDocument.body.parentElement.innerHTML = document.body.parentElement.innerHTML`, then fill the form and submit. (Updated the example code.) Good luck! – Michael G Feb 16 '23 at 11:55
  • I have analyzed deeply your proposal. It would have been the way to go, but the .click() from the inside the iframe behaves weirdly. Specifically, if I use two inputs, the first time it doesn't do anything while the second time it does something (not completely but mostly). I understand it's impossible for you to help further without seeing what I'm seeing, for the time being I'll simply upvote the answer but I'll accept it if I manage to use it in the end :) – Matteo NNZ Feb 16 '23 at 14:30
  • I got inspired by what you proposed, but instead of using an iframe in the same page, I've opened a new tab. You can check the answer I posted. If you embed the content of that answer in yours, I will accept yours and delete mine (since in the end it's you who moved me to the right way :)) – Matteo NNZ Feb 17 '23 at 16:16
  • What?! No, no, no. That was *not* my idea and I have *no* idea why your code works. `about:blank` is a different domain, guaranteed. Doesn't this fly in the face of everything domain isolation is meant to prevent? :-| (But I will definitely experiment with your discovery, so thank you for posting it.) – Michael G Feb 17 '23 at 21:10
  • As I said, I have no idea about what I did, I just mixed up some answers after getting inspired by what you proposed (creating an iframe and posting from there), except that instead of creating an iframe I opened a new tab in which I copied my main page source. I wouldn't be surprised that the website I'm interacting with has no domain isolation enforced, and that is maybe why it works. Anyway, I've accepted my answer (since it was what actually worked) but thanks a lot for the tip! – Matteo NNZ Feb 20 '23 at 08:34
0

In case it may help someone in the future, I finally was able to work around the problem by opening a new tab (and working in that tab) per iteration of my loop.

Something like this:

while (inputs.length > 0) {
    const singleInput = inputs.pop();
    const newWindow = window.open('about:blank', '_blank');
    newWindow.addEventListener('load', () => {
        newWindow.document.body.parentElement.innerHTML = document.body.parentElement.innerHTML;
        fillForm(newWindow.document, singleInput) //<-- the function fill form uses the document in parameter to perform the different get/set
        newWindow.document.getElementById("submit-button").click();
    });
}
Matteo NNZ
  • 11,930
  • 12
  • 52
  • 89