How to configure httpc profiles with rebar3?

Question

How can I set configuration options for httpc's profiles when using rebar3?

Here is the only example being via erl -config inets.config that looks like this:

[{inets, 
[{services,[{httpc,[{profile, server1}]},
            {httpc, [{profile, server2}]}]}]
}].

I tried adopting it to my rebar3 project structure.

Code

Project was created with rebar3, with standard OTP layout:

rebar3 new release myapp

Here is my myapp/config/sys.config:

[
  { myapp, []},
  {inets, [{services, [{httpc, [{profile, myapp}]}]}]}
].

rebar.config:

{erl_opts, [debug_info]}.
{deps, []}.

{relx, [{release, { myapp, "0.1.0" },
         [myapp,
          sasl]},

        {sys_config, "./config/sys.config"},
        {vm_args, "./config/vm.args"},

        {dev_mode, true},
        {include_erts, false},

        {extended_start_script, true}]
}.

{profiles, [{prod, [{relx, [{dev_mode, false},
                            {include_erts, true}]}]
            }]
}.

Here is my myapp.app.src file for completeness:

{application, myapp,
 [{description, "An OTP application"},
  {vsn, "0.1.0"},
  {registered, []},
  {mod, { myapp_app, []}},
  {applications,
   [kernel,
    stdlib
   ]},
  {env,[]},
  {modules, []},

  {maintainers, []},
  {licenses, []},
  {links, []}
 ]}.

Requests

Here is a request I'm trying to make from rebar`s shell:

$ ./rebar3 shell
1> ===> Booted myapp
1> ===> Booted sasl
...
1> httpc:request( "http://reddit.com", myapp).
** exception exit: {noproc,
                    {gen_server,call,
                     [httpc_myapp,
                      {request,
                       {request,undefined,<0.88.0>,0,http,
                        {"reddit.com",80},
                        "/",[],get,
                        {http_request_h,undefined,"keep-alive",undefined,
                         undefined,undefined,undefined,undefined,undefined,
                         undefined,...},
                        {[],[]},
                        {http_options,"HTTP/1.1",infinity,true,
                         {essl,[]},
                         undefined,false,infinity,...},
                        "http://reddit.com",[],none,[],1478280329839,
                        undefined,undefined,false}},
                      infinity]}}
     in function  gen_server:call/3 (gen_server.erl, line 212)
     in call from httpc:handle_request/9 (httpc.erl, line 574)

Here is the request without a profile, to check that inets actually works:

2> httpc:request( "http://reddit.com").

=PROGRESS REPORT==== 4-Nov-2016::13:25:51 ===
          supervisor: {local,inet_gethost_native_sup}
             started: [{pid,<0.107.0>},{mfa,{inet_gethost_native,init,[[]]}}]

=PROGRESS REPORT==== 4-Nov-2016::13:25:51 ===
          supervisor: {local,kernel_safe_sup}
             started: [{pid,<0.106.0>},
                       {id,inet_gethost_native_sup},
                       {mfargs,{inet_gethost_native,start_link,[]}},
                       {restart_type,temporary},
                       {shutdown,1000},
                       {child_type,worker}]
{ok,{{"HTTP/1.1",200,"OK"},...

please show your `rebar.config`. – reith Nov 04 '16 at 14:15 — reith, Nov 04 '16 at 14:15

reith · Accepted Answer · 2019-05-18T17:04:56.770

rebar3 itself use inets http clients so when It starts your application in shell, inets is already started and configured. One workaround would be stop inets before your application starts, as It's suggested by rebar3 developer (copied below). Another one would be boot your release in console mode:

./_build/default/rel/myapp/bin/myapp console

Beside that there is another problem with your project. You have not told you want inets being started for you. You should have this line in myapp.src:

{applications, [kernel, stdlib, inets]}

Or you can list inets in rebar.config release section, to tell relx this app should be included in release and started on boot.

{relx, [{release, { myapp, "0.1.0" }, [inets, myapp, sasl]} ]}

Stop Inets from Loading on rebar3 shell startup

Here is the copy of the full answer by Fred Hebert from the rebar3 mailing list:

We do need inets for package fetching and will likely not turn it off automatically for all use cases as this could compromise general usage of the rebar3 agent in case where the user application does not use inets, but still asks to fetch packages in a subsequent run. The workaround I would suggest would be to use a hook script for it. Hook scripts run before we boot user applications and are regular escripts:
#!/usr/bin/env escript 
main(_) -> application:stop(inets). 
You can then hook the script in with:
{shell, [{script_file, "path/to/file"}]} 
in rebar3.config, or with
rebar3 shell --script_file test/check_env.escript

Thank you. You were right about inets starting automatically with the rebar3 shell! Doing the little trick in the link helped. — Andriy Drozdyuk, Nov 07 '16 at 18:46

score 0 · Answer 2 · edited Nov 03 '16 at 21:40

I can't find anything in the documentation to suggest that rebar.config can contain the application configuration you want.

Instead, application configuration is often kept in a configuration directory in your application, and like the example you gave says, you must use the -config flag when launching erl to point to the configuration file.

If you use rebar3 for making release and start your service by script made with relx (from _build/default/rel/<release_name>/bin/<release_name> by default) the application configuration file is passed to erl for you. If config file exist in your application directory, by default in config/sys.config, It will be regarded as application configuration, otherwise an empty configuration will be made. You can customize Its path by relax' release option sys_config.

For our software, we typically had a single config/sys.config file. The structure is the same as the config sample you have provided. Note that the configuration file can contain settings for many different services. For example, mixing inets with one of ours:

[{inets, [
     {services,[{httpc,[{profile, server1}]},
     {httpc, [{profile, server2}]}]}
 ]},
 {my_app, [
     {my_setting, "my value"}
 ]}
].

We launch with erl -config config/sys.config. This way if we need to set service configuration we can simply update our configuration file, which also houses the configuration specific to this application.

As far as I'm aware, this is the correct method to use. I have not been able to find documentation supporting any other way of doing this.

Don't know who downvoted this. Sorry mate. Thanks for suggestion, but it's not quite what I'm looking for. — Andriy Drozdyuk, Nov 03 '16 at 18:36
Reith has expanded on the answer. Perhaps now it helps out more? (Thanks Reith!) — Andrakis, Nov 03 '16 at 21:40
Still doesn't work. I'm getting `noproc` when I try to issue a `httpc:request` with the `myapp` profile. I tried the `httpc` entry standalone like you suggested as well. — Andriy Drozdyuk, Nov 04 '16 at 17:03
You need to start the inets process using `inets:start().` If you wish to use SSL, you also need to do `ssl:start().` Then you can make `httpc` requests. Note that this can be accomplished automatically if you add it to your `.app` profile. See [this StackOverflow](http://stackoverflow.com/questions/1839862/erlang-when-to-perform-inetsstart) question for more info on doing that. — Andrakis, Nov 06 '16 at 21:50
The issue was the rebar3 shell starting inets automatically. See accepted answer. Thanks for looking into this anyways! — Andriy Drozdyuk, Nov 07 '16 at 18:50

How to configure httpc profiles with rebar3?

Code

Requests

2 Answers2

Stop Inets from Loading on rebar3 shell startup