5

I have a role, that has to create a list of directories. Which directories end up in the list, depends on several conditions that are evaluated at runtime.

When I use the file module, like shown, this is very slow. Creating and even checking for the existence of every directory takes about half a second, which easily sums up to minutes in the scenarios I encounter.

Pipelining is enabled.

- name: Create directories
  file:
    path:   "{{ item }}"
    state:  directory
  with_items:
    - "{{ dirs }}"
  when:
    - dirs is defined

A similar question has been asked here, however using synchronize or unarchive, as suggested there, seems very awkward for directories not known in advance, as the directory structure to be synchronized, first has to be created on the local host somewhere.

Are there other alternatives to solve this, I might have missed?

EDIT: I am aware of shell and command and mkdir -p happens to be idempotent. Still I would prefer a way, where ansible manages the state of the directories.

nlu
  • 1,903
  • 14
  • 18
  • I experienced the same problem having to generate a list of directories based on `find` module. I've not read the code so what I'll wrote is just educated-guess. The slowness is related to the executor to have for each directory, connect to the target and authenticate, push the python generated code to the client, check in the directory is not yet created with the expected stated, if not, create it. report to the executor which will gather the info and so forth. – Baptiste Mille-Mathias Sep 10 '18 at 14:09
  • @Baptiste Mille-Mathias In the question I linked to, the first answerer drew the same conclusion. – nlu Sep 10 '18 at 14:40

1 Answers1

2

An option would be to use the "command creates" construct.

- name: Create directories
  command: mkdir -p "{{ item }}"
  args:
    creates: "{{ item }}"
  loop: "{{ dirs }}"

Idempotency, command module and "tools no policy".

Both in the comments to this answer and in the thread mentioned in the question idempotency of the command module is discussed and native idempotency of ansible modules preferred.

" I am aware of shell and command and mkdir -p happens to be idempotent. Still I would prefer a way, where ansible manages the state of the directories."

"You should mention that this loose idempotent feature of using native file module."

The command creates construct is idempotent. The command is executed only if the item does not exist. Moreover, in my opinion, this construct shall be preferred in similar situations because it does exactly what is needed, does it fast and is easy to understand. This complies with the norms of minimalist, modular software development which are necessary for sustainable development.

Community
  • 1
  • 1
Vladimir Botka
  • 58,131
  • 4
  • 32
  • 63
  • You should mention that this loose idempotent feature of using native `file` module – Baptiste Mille-Mathias Sep 12 '18 at 08:57
  • @Vladimir Botka I do agree with your explanation now. Sorry for not getting your point earlier, but I actually thought of a plain `mkdir -p {{ dirs }}`. It will take me a while now to test, if your solution actually alleviates the mentioned performance problem. – nlu Sep 13 '18 at 14:28
  • It takes me the same amount of time as file module takes, waiting few second for every created directory. – BartBiczBoży Aug 21 '19 at 16:06