Autofill the title of website via link in a form

Question

<form method="POST" action="{{ route('storeCompany') }}">
            @csrf
            <label>{{ __('Website URL') }}</label>
            <input type="text" name="url" value="{{ old('url') }}" placeholder="{{ __('http://example.com') }}" class="form-control" required="required">
            <label>{{ __('Website Title') }}</label>
            <input type="text" name="name" value="{{ old('name') }}" placeholder="{{ __('Example Ltd') }}" class="form-control" required="required">
            <input type="submit" name="sbNewReviewItem" class="btn btn-block btn-primary" value="{{ __('Submit New Company') }}">
</form>

I want the "Website Title" field to be auto-filled when the user types the "Website URL" also "Website Title" must be editable if the user want to. How can I do this? Please help :(

Example: When the user enters https://stackoverflow.com/ URL in the "Website URL" field "Website Title" filed must auto-filled as Stack Overflow - Where Developers Learn, Share, & Build Careers

Assuming you're talking about the title which appears in the browser bar when you view the site's home page, you'd have to make a http request to the URL, parse the html that's returned and extract the contents of the title element from it. Probably not very practical as the user is typing. Unless you have a pre defined list of URLs and titles in your own database? (But then if you did, you'd be using a drop-down list or autocomplete control instead of a bare text box) — ADyson, Jul 05 '21 at 12:24
Please explain what you mean by "Website Title" and "Website URL". Is it a model property? — waterloomatt, Jul 05 '21 at 12:24
As ADyson alluded to, this can be quite a complicated task. You need to think about what will trigger the "fetch". Will be on on every keypress? Will you run the URL through some regex first to validate the user hasn't entered garbage. Once you have the HTML payload, you need to parse it to extract the __ tag. This may or may not be a trivial task. Can you tell us what you've tried so far or how you envision it to work? — waterloomatt, Jul 05 '21 at 12:57
https://stackoverflow.com/questions/7901760/how-can-i-get-the-title-of-a-webpage-given-the-url-an-external-url-using-jquer contains some suggestions about how to do it — ADyson, Jul 05 '21 at 13:28

dreamLo · Accepted Answer · 2021-07-19T10:08:14.263

2

You have a change event on the url-field element which does a fetch request to an api website that returns the html contents of that website, and then it uses regex to extract the html title and then it adds it to the name field.

Also while the fetching is executing we show a Fetching text to let the user know what's going on. And the form submit event has been stopped with the preventDefault() function. Here you can do what ever you want after the user submits the form.

Instead of using https://api.codetabs.com/v1/proxy/?quest= which will limit your requests and show you a Too many requests error, you should use your own code that fetches the website contents and then return the response. To optimize this you can return just the website title, to save on bandwidth and cpu cycles.

The reason you need an api to fetch the website content is because of CORS. You can't just use fetch resources from any website you want. You can only do this from the current website or from websites that have allowed you to do so.

document.addEventListener('submit', event => {
  event.preventDefault()
})

document.addEventListener('change', async (event) => {
  if (event.target.classList.contains('url-field')) {
    await changeUrl(event.target.closest('form').elements)
  }
})

async function changeUrl (fields) {
  try {
    if (!fields.url.value) return
    fields.output.removeAttribute('hidden')
    const response = await fetch('https://api.codetabs.com/v1/proxy/?quest=' + encodeURI(fields.url.value))
    const html = await response.text()
    const match = /<title[\s\S]*?>([\s\S]*?)<\/title>/gi.exec(html)
    if (!match || !match[1]) throw new Error('No title found')
    fields.name.value = match[1].trim()
    fields.output.setAttribute('hidden', '')
  } catch (error) {
    console.error(error)
  }
}

<form method="POST" action="">
<p>
  <label>Website URL</label>
  <input type="url" name="url" value="" placeholder="http://example.com" class="form-control url-field" required>
  <output name="output" hidden>Fetching</output>
</p>
<p>
  <label>Website Title</label>
  <input type="text" name="name" value="" placeholder="Example Ltd" class="form-control">
</p>
<input type="submit" name="sbNewReviewItem" class="btn btn-block btn-primary" value="Submit New Company">
</form>

edited Jul 19 '21 at 10:08

answered Jul 13 '21 at 20:34

dreamLo

1,612
12
17

looks good, two remarks: 1. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/encodeURIComponent to ensure the proxy API request always works and 2. as it is in the browser, I think it has a better way to parse HTML than regex, e.g. DOMDocument and getTagByName ;) – hakre Jul 16 '21 at 16:42
1) I have added `encodeURI` to the url parameter, thanks. 2) You don't want to use DOMDocument to parse the document because the website can be extremely large and you are using resources for something you don't need. You only need to extract the title tag from that text, not parse it and fetch all sorts of data. Regex in this case is very fast compared to the dom parsing. – dreamLo Jul 16 '21 at 19:49
That sounds insightful. But still I can imagine that the browsers parser for HTML is more effective than the regex (like in theory), I don't know it, but if using partial content (e.g. the first 4096 bytes or so) it could further optimize it (same for the regex approach). Maybe that's a sweet spot. All in all: This were just remarks, no request to edit the answer, more FYI (can't upvote your comment anymore today, sorry). – hakre Jul 16 '21 at 20:04
The browser dom is more efficient than regex if you want to extract everything. But if you only need one thing, then regex wins hands down. – dreamLo Jul 16 '21 at 20:18
well not so sure. if you would need to parse the HTML with regex from the beginning of the string, I'm not sure it's faster. the regex here for example does not address whitespace in the opening title tag for example (nor attributes). For sure that one wins (perhaps a string search is even faster, seeing the pattern, totally doable as well for the closing tag search). – hakre Jul 16 '21 at 20:25
@dreamLo It works! but with some sites, It doesn't. Eg; https://www.binance.com/ – Jul 17 '21 at 23:16
@NortonVUV That's because of the regex, it didn't take into account ``. I fixed the regex, so now `binance.com` also works. – dreamLo Jul 18 '21 at 08:48
@dreamLo please check this URL as well: ```digital-investment.org``` – Jul 18 '21 at 12:17
Now the regex should work for any type of white space inside the `` and there's also error handling for when the title simply doesn't exit on the page. – dreamLo Jul 19 '21 at 10:09
@dreamLo tysm for helping ++ – Jul 20 '21 at 10:15

score 2 · Answer 2 · answered Jul 14 '21 at 09:08

Because you're example look like Laravel. I found it appropriate to write an answer in PHP.

example.blade.php

<title>{{ $title ?? 'Default title' }}</title>
<form method="POST" action="{{ route('storeCompany') }}">
    @csrf
    <label>{{ __('Website URL') }}</label>
    <input type="text" name="url" value="{{ $url ?? old('url') }}" placeholder="{{ __('http://example.com') }}" class="form-control" required="required">
    <label>{{ __('Website Title') }} <small>(Leave blank to retrieve from URL)</small></label>
    <input type="text" name="name" value="{{ $title ?? old('name') }}" placeholder="{{ __('Example Ltd') }}" class="form-control">
    <input type="submit" name="sbNewReviewItem" class="btn btn-block btn-primary" value="{{ __('Submit New Company') }}">
</form>

CompanyController.php

<?php

namespace App\Http\Controllers;

use Illuminate\Http\Request;
use Illuminate\Support\Facades\Http;

class CompanyController extends Controller
{
    public function index(){
        
        return view('example');
    }
    
    public function storeCompany(Request $request){
        
        $title = $request->name;

        if (empty($title) && filter_var($request->url, FILTER_VALIDATE_URL)){
            $response = HTTP::get($request->url);
            if ($response->ok()){
                preg_match('/<title>(.*)<\/title>/i', $response->body(), $matches);
                $title = $matches[1] ?? null;
            }
        }

        $data = [
            'url' => $request->url,
            'title' => $title
        ];
        
        return view('example', $data);
    }
}

Keshav Bajaj · Answer 3 · 2021-07-14T12:52:40.343

0

Here is how to:

Read the website url.
Send an ajax request to codetabs api to get full html.
Get the title from the html.
Do whatever you need with the title.

function gettitle(website) {
  if(website.trim() == ''){
   return false;
  }
  var url = 'https://api.codetabs.com/v1/proxy/?quest=' + website;
  $.ajax({
    url: url,
    success: function(responseHtml) {
      var newTitle = $(responseHtml).filter('title').text();
      document.getElementById('title').innerHTML = 'Title: ' + newTitle
    },
    error: function() {
      document.getElementById('title').innerHTML = newTitle = 'Sorry that page doesn\'t';
    }
  });
}

<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
<input placeholder='Website' oninput='gettitle(this.value)' type='url' id='url'>
<div id='title'></div>

edited Jul 14 '21 at 12:52

answered Jul 13 '21 at 04:52

Keshav Bajaj

863
1
5
13

Most sites you try to call will block this Ajax-based approach because they won't allow CORS requests. It would have to be done server side to be reliable – ADyson Jul 13 '21 at 06:57
You can try running it with various websites. It works with most of them. – Keshav Bajaj Jul 13 '21 at 07:03
I would suggest that "most" isn't really reliable enough. – ADyson Jul 13 '21 at 09:16
Please mention some of the websites which aren't working so, I can improve my code. – Keshav Bajaj Jul 13 '21 at 17:36
This is highly inefficient to place a website's content in a created div block because depending on the website it could take a while for the browser to process it. It's better to use regex and parse the html. – dreamLo Jul 14 '21 at 07:19
@ADyson Depending on this application usage, sending a server-server request has its limitations too, if number of users who enter a certain website increases, there will be a huge number of requests sent from server to that website, so the IP of server will be blacklisted. There are other parameters included too for example privacy issues. Generally speaking handling this on server-side is better than on the front but there is no best solution for this :) – Mohsen Nazari Jul 16 '21 at 00:01

score 0 · Answer 4 · answered Jul 13 '21 at 07:13

As soon as user has entered the data.

Get the value using $("#text_field").val();

Have an ajax call and get the value using the below function, pass the url to t

require 'simple_html_dom.php';
function dataParser($url){
    $html = new simple_html_dom();
    $html->load_file($url); 
    $title = $html->find('title',0)->innertext;
    return $title;
}
// something like $_GET['user_url'];
echo  $title = dataParser('https://stackoverflow.com');

this code will output the title, set it to the div where you want to show the title, something like this

var ajaxCallResponse = ""//store the response here
$("#title_div").val(ajaxCallResponse);

Autofill the title of website via link in a form

4 Answers4