Outputing analog voltage continuously to NI DAQ modules with nidaqmx-python

Question

This is about working with the nidaqmx-python package, maintained by National Instruments for the purpose of interfacing their acquisition modules.

Specs: NI cDAQ-9178 with NI 9264 output card plugged into it. Package nidaqmx-python in a conda virtual environment for Python 3.7 on a machine running Windows 10.

Overarching goal: read an input voltage continuously and, after lowpass filtering it in real time with an IIR filter, output some PID-calculated analog voltage to continuously drive some machine (irrelevant which one).

Concrete goal right now: understand how to make the best possible use of nidaqmx-python high-level functions and callbacks to continously output voltage through my cDAQ in a way that is efficient, with the PC buffer being correctly managed and all, and understanding how this happens.

Knowledge: I'm OK in python but I have only been playing with the nidaqmx-python package for some weeks now. I have successfully managed to use the built in callback mechanism which allows to continuously read an analog signal at some sampling rate, and thought it would then be straightforward to do the writing part. It seems it is not and I am struggling with it, although I have read the (not very friendly?) documentation for the package, here.

Issues: with the code here below, which seemed to be like a good and easy way to try to get to know these functions, I simply try to increase the values in an array, data, representing the voltage to be outputed, and then with the function register_every_n_samples_transferred_from_buffer_event (documented here) I have the callback my_callback be called every time the device has read 10 samples from the PC buffer. That callback does something simple: it uses write_many_sample to write data to the PC buffer. I wanted to use this simple example to check if, with these parameters, I could get from 0 to 5V in 5 seconds (seeing as I increase by 0.01 Volts every 10 ms, because the rate is 1000 Hz and the callback is called every 10 transferred samples, i.e. at 100 Hz). This fails and I go from 0 to 5 Volts in approximately 25 seconds (checked with a mutlimeter).

Code:

# Continuous write single channel
import numpy as np

import nidaqmx
from nidaqmx.stream_writers import (AnalogSingleChannelWriter)
from nidaqmx import constants

global datasize
global bufsize
global rate_outcfg
global rate_callback
datasize = 10  # I guess this ought to be same as rate_callback
bufsize = 10  # issues warnings as is; can be increased to stop warnings
rate_outcfg = 1000  # chosen at random; contraint is to update output at 100Hz so whatever works would be fine here
rate_callback = 10  # basically rate_outcfg/100 as I would like to update output at 100Hz (note setting it to that does not work)

# ISSUE: it seems instead of refreshing voltage every second it updates every bufsize/10 seconds, if counter_limit = 100
# This means there is something I am missing

global counter_limit
counter_limit = 1  # 1 to update every callback call (which is supposed to be at 100Hz rate)

global data
data = np.empty((bufsize,))  # cannot be vertical for nidaqmx to work
data[:] = 0  # starting voltage in Volts

global stream

global counter
counter = 0

def my_callback(task_idx, event_type, num_samples, callback_data):
    global counter
    global counter_limit

    if counter == counter_limit:  # with 100, voltage will change at 1Hz given the above parameters (should be config better)
        counter = 0
        data[:] = data[:] + 0.01
    else:
        counter = counter + 1

    stream.write_many_sample(data, timeout=constants.WAIT_INFINITELY)
    return 0

def setTask(t):
    t.ao_channels.add_ao_voltage_chan("cDAQ2Mod8/ao0")
    t.timing.cfg_samp_clk_timing(rate=rate_outcfg, sample_mode=nidaqmx.constants.AcquisitionType.CONTINUOUS,
                                    samps_per_chan=bufsize)  # last arg is the buffer size for continuous output

task = nidaqmx.Task()
setTask(task)

stream = AnalogSingleChannelWriter(task.out_stream, auto_start=False)  # with auto_start=True it complains

# Call the my_callback function everytime rate_callback samples are read by device from PC buffer
task.register_every_n_samples_transferred_from_buffer_event(rate_callback, my_callback)

stream.write_many_sample(data)  # first manual write to buffer, required otherwise it complains it can't start

task.start()
input('hey')  # task runs for as long as ENTER is not pressed
task.close()  # important otherwise when re-running the code it says specified device is reserved!

# NOTE somehow once ENTER is pressed it takes some seconds to actually stop if bufsize is very large, I don't know why

Notes:

• It seems the timing depends on the bufsize: doubling it to 20 results in reaching 5 Volts in approximately 50 seconds instead of 25 s.
• Unless I make my buffer very large, I get a warning at every callback call:

While writing to the buffer during a regeneration, the actual data generated might have alternated between old data and new data. That is, while the driver was replacing the old pattern in the buffer with the new pattern, the device might have generated a portion of new data, then a portion of old data, and then a portion of new data again.
Reduce the sample rate, use a larger buffer, or refer to documentation about DAQmx Write for information about other ways to avoid this warning.
  error_buffer.value.decode("utf-8"), error_code))
C:\Users\james\anaconda3\envs\venv37_drift_null\lib\site-packages\nidaqmx\errors.py:141: DaqWarning: 
Warning 200015 occurred.

• Note the counter variable counter to allow for some flexibility (currently counter_limit is set to 1 so that the output data increases every run).

Bottom line: I'm a bit lost with this. Ideally I would like to understand how I can achieve, e.g. going from 0 to 5 V in 5 seconds. But this is only an example. I would like to understand what role is played by the different variables bufsize, rate_callback and rate_outcfg and what sets the timing of execution. Ultimately I'd like to get to the point where my basic understanding enables me to write such a simple task (outputing a continuously increasing voltage - or some other function like a sine wave) in a way that is efficient and warning free).

Many thanks to anyone contributing !