3

The following is standard code, adding tasks to a queue, and then the worker pool will process the tasks. What I am confused about is what it means to have more than 1 worker in the queue, as I don't believe this code will start more than one node.js process? 

var Queue = require('firebase-queue');
var firebase = require('firebase');

firebase.initializeApp({
    // propriety data
});


var promise = firebase.auth().signInWithEmailAndPassword(process.env.EMAIL, process.env.PASSWORD);

promise.then(function () {
    return createQueue();
}, function (error) {
    console.error(error);
});

var queue;

function createQueue() {

    var db = firebase.database();
    var ref = db.ref('quote-request-queue');
    var tasks = db.ref('quote-request-queue/tasks');

    tasks.push({'foo': 'bar'});
    tasks.push({'foo': 'bar'});
    tasks.push({'foo': 'bar'});

    const opts = {

    };

    queue = new Queue(ref, opts, function (data, progress, resolve, reject) {
        // Read and process task data
        console.log(data);

        // Do some work
        progress(50);

        // Finish the task asynchronously
        setTimeout(function () {
            resolve();
        }, 100);
    });

    console.log('worker count:', queue.getWorkerCount());
}

my question is - is it necessary to call resolve() before the next item in the queue will be processed by another worker?

I want to find out how to make workers block each other (for the purposes of DB atomicity) as well as to have workers work in parallel on separate tasks. (They will work in parallel/concurrently if there is I/O happening in the processing function).

How can I control blocking/non-blocking worker queues with Firebase?

Alexander Mills
  • 90,741
  • 139
  • 482
  • 817
  • perhaps the number of workers determines somehow the maximum number of units of work that could possibly run in parallel...? – Alexander Mills Oct 04 '16 at 00:05
  • looks like in order to get N Node.js processes, we have to simple start this script N times. But that still doesn't answer the question of what more than 2 workers gives us, within a single process. – Alexander Mills Oct 04 '16 at 00:09
  • after playing around with this code, if a single node.js process is used to run this code and 1 worker is selected, then the code will block, one task can start only once the previous has completed (resolve() is called). – Alexander Mills Oct 04 '16 at 02:27
  • So @Alexander, I have this same question, have you determined what the purpose of more than 1 worker is? – aez Oct 22 '16 at 12:26
  • no I havent, I think it has to do with single threaded concurency - how many asynchonous operations can happen in parallel (interleave). with 1 worker that should mean that an async operation has to complete before the next one starts. all in all, its an abuse of language - "worker" should be reserved for a separate thread or process, it has no business being used in the context I believe it used here... – Alexander Mills Oct 22 '16 at 18:35

0 Answers0