How to manage dependencies in Perl ABOVE package level and WITHOUT focussing on CPAN?

Question

This is related to a former question about CPAN::Meta::Spec, which doesn't seem to be designed for what I thought I could use it.

My use case

I have different Perl apps containing of a lot of packages themself, depending on some home grown system wide utility stuff and of course 3rd party packages. Those apps might provide different features depending on runtime config and depending on those features, dependencies are different. Additionally, if automated tests are available, those might introduce dependencies as well and in the most current example those dependencies where available in ActiveState Perl 5.22 on Windows by default already, where on Ubuntu 16.04 those needed to be installed using APT.

My own apps and utility stuff are maintained using Subversion and the repos at least partly publicly available to customers as well. 3rd party packages are preferred to be maintained by the package manager of the OS like APT or the Perl distribution, like PPM for ActiveState Perl on Windows. Only if some dependency is not available that way, CPAN comes into play.

What I would like to achieve...

...is describing my apps and system wide utility stuff with their dependencies in a way that allows me to distinguish e.g. runtime from development or tests and being able to distinguish optional features and their dependencies. Additionally, I would like to maintain some repos where e.g. my own apps and system wide utilities can be downloaded from.

What I don't want...

...is to maintain at multiple places each and every individual package I'm using somewhere, because that doesn't make much sense to me. APT is not necessarily installing individual Perl packages, but instead some higher level distribution containing multiple of those, a complete app for example. If dependencies are introduced, developers need to check anyway if those are available on all platforms of interest, which is not necessarily the case, and how those are distributed on each platform, e.g. APT, PPM or CPAN. So dependencies on individual packages can not be introduced easily without checking how those packages are distributed on which platform. Therefore it seems enough for my needs to manage dependencies ABOVE a package level, which is what mostly gets maintained on platforms anyway.

Additionally, this is how e.g. Maven and Gradle work: One doesn't depend on individual classes like org.apache.commons.lang3.AnnotationUtils or org.apache.commons.io.ByteOrderMark, but on distributions containing those classes like Apache Commons Lang and Apache Commons IO in some specific version. While individual classes are imported in some class in the end, the project description and its dependencies itself doesn't contain that level of detail. I don't see why I should dive deeper in Perl if it works for Java pretty well and if I need to check at distribution level anyway if Perl-dependencies can be fulfilled at all.

What I need

So, what I need is some spec/DSL to describe my project with some name, version number, its dependencies and most likely different repos where to get things from. Such a repo would be my own SVN-repos or concepts like APT or PPM, at best even with additional repos for those, if one decides to host such. In the end, some tools like Ansible should be used to install some app of mine while being able to deal with dependencies automatically, using plugins or additional tools or whatever, based on the specs in each app/project.

What I've found so far

For Perl this lead me to cpanm, cpanfile and CPAN::Meta::Spec, both being able to distinguish runtime from test, support optional features etc. The latter can even model different repos. But both only seem to support dependencies on package instead of some higher distribution level. One might be able to work around this using some packages acting as placeholder for distributions, though. E.g. by creating sysutils.pm containing package sysutils; and defining some version only. Apps could then depend on that package using the above formats.

Downsides recognized so far

But people e.g. recommend to not write CPAN::Meta::Spec manually, but use some build and authoring tools instead. The problem with those is that some/most are considered deprecated by the linked blog post itself or other sources and some examples even are providing manually written CPAN::Meta::Spec to those tools in the end only. And if not, they sometimes simply expect infos of CPAN::Meta::Spec in some custom configuration format, which seems not as well defined as the spec itself. So why use those tools at all instead of writing the spec manually?

Even the CPAN::Meta::Spec itself seems to be problematic, as the most current version 2 seems to prefer JSON for some undocumented reason. That is a bad decision for manually writing and maintaining that spec of course, because JSON lacks comments, which is a no-go for any kind of description maintained by humans in my opinion.

Questions

1. So, which format/spec can I use to manually describe some abstract distribution using some name and version, including it's optional features and dependencies? Pretty much what seems to be possible with CPAN::Meta::Spec already, only that package level dependencies seem too low level for me currently.

2. Which tool should be used to resolve dependencies based on the former spec and supports different source-repos for dependencies like APT, PPM and SVN?

3. With things like Ansible coming into play, is it worth it to maintain some Perl-specific description at all anymore or should one simply use Ansible to describe dependencies and let Ansible provide all of those using plugins or whatever?

Thanks for your suggestions!

Answer 1

I want to model dependencies on what CPAN::Meta::Spec calls a distribution

When using ExtUtils::MakeMaker, META_MERGE gives you access to fields from the META spec that allow you to specify the information you want to specify. The following snippet from DateTime::Format::Atom demonstrates this:

META_MERGE  => {
   'meta-spec' => { version => 2 },

   prereqs => {
      configure => {
         requires => {
            'ExtUtils::MakeMaker'       => 6.74,
         },
      },
      runtime => {
         requires => {
            'strict'                    => 0,
            'version'                   => 0,
            'warnings'                  => 0,
            'DateTime'                  => 0,
            'DateTime::Format::RFC3339' => 0,
         },
      },
      test => {
         requires => {
            'Test::More'                => 0,
         },
      },
      develop => {
         requires => {
            'FindBin'                   => 0,
            'Pod::Coverage'             => 0.18,
            'Test::Pod::Coverage'       => 1.08,
         },
      },
   },
},

Consult this for the full prereqs spec.

---

But both only seem to support dependencies on package instead of some higher distribution level."

You're making a fuss about nothing. It doesn't matter that you have to use Foo::Bar instead of Foo-Bar.