Programmatically alter Elixir Code

Question

I need to load, alter and write the code in a mix.exs file. I want to be able to load the file, write the dependencies and write the file.

I start with:

defmodule Elixir_2ndTest.Mixfile do
  use Mix.Project

  def project do
    [app: :elixir_2nd_test,
     version: "0.0.1",
     elixir: "~> 1.2",
     build_embedded: Mix.env == :prod,
     start_permanent: Mix.env == :prod,
     description: description(),
     deps: deps]
  end

  def application do
    [applications: [:logger]]
  end

  defp deps do
    []
  end
end

And I need to end up with (the only difference is in the deps fun):

defmodule Elixir_2ndTest.Mixfile do
  use Mix.Project

  def project do
    [app: :elixir_2nd_test,
     version: "0.0.1",
     elixir: "~> 1.2",
     build_embedded: Mix.env == :prod,
     start_permanent: Mix.env == :prod,
     description: description(),
     deps: deps]
  end

  def application do
    [applications: [:logger]]
  end

  defp deps do
    [{:httpoison, "~> 0.8.3"}]
  end
end

The dependencies come from a different build system (I cannot use hex directly form the public internet, so I use it in OFFLINE mode and drop the dependencies in .hex/

I know what teh depenencies and what the versions are an need to insert them in the deps function (in this case httpoison 0.8.3).

If my understanding is correct this should be possible by loading the file, quoting, altering, unquoting.

This is what I have up until this point:

{:ok, body} = File.read("mix.exs")
{:ok, ast} = Code.string_to_quoted(body)

Any pointer on how I can alter the ast and write it back would be appreciated.

I looked into this a little bit in hopes of adding something like `mix install` similar to `npm install`, you'll basically have to parse and lex the file. I haven't had a chance to dig into that. If you do it would be great, but its definitely non-trivial since deps could be a function, or multiple functions, or a list in the project function. — TattdCodeMonkey, Jul 01 '16 at 22:18
I know what the deps are. I just need to drop them in there. Anything that's in there would get replaced. — Mircea, Jul 01 '16 at 22:34

score 1 · Answer 1 · answered Jul 01 '16 at 22:18

It won't look exactly the same, but you can use Macro.to_string to convert the ast back to elixir code.

I was playing around with using my library PhStTransform to modify the ast and convert it back to code. Here's a very simple example from the PhStTransform test library.

test "transform quote do output" do
  data = quote do: Enum.map(1..3, fn(x) -> x*x end)
  data_transform = quote do: Enum.map(1..3, fn(y) -> y*y end)
  replace_x = fn(a, _d ) ->
    case a do
      :x -> :y
      atom -> atom
    end
  end
  potion = %{ Atom => replace_x }
  assert PhStTransform.transform(data, potion) == data_transform
end

What that does is convert all references to :x in the ast into :y. You'd need to be a bit more clever with writing the potion for PhStTransform, but I think it should be possible. PhStTransform is in hex.pm.

https://hex.pm/packages/phst_transform

score 1 · Answer 2 · edited May 23 '17 at 12:33

1

I'm not an Elixir expert, but I know about transforming source code; see my bio.

If you have access to the AST as a data structure, you can always write procedural code to climb over it and hack at where you want something different. I assume if Elixir will give you the AST, it will give you access/modification procedures for working with it. This is compiler 101.

That's usually NOT pretty code to write or maintain. And, it may not be enough: you often need more than just the AST to do serious analysis and transformation. See my essay on Life After Parsingl. Think of this as compiler 102.

One the first stumbling blocks is regenerating text from the AST. Here is my SO discussion on how to prettyprint an AST, and why it is harder than it looks: https://stackoverflow.com/a/5834775/120163

(Sounds like Fred the Magic Wonder Dog didn't think what Elixir offered was enough and is inventing his own extensions to make this easier.).

edited May 23 '17 at 12:33

Community

1
1

answered Jul 01 '16 at 23:11

Ira Baxter

93,541
22
172
341

yeah. sort of get how this is supposed to work, and wondering if there is something that can be done for Elixir. – Mircea Jul 01 '16 at 23:42
Elixir AST is an Elixir data structure so it's very easy to transform. There aren't a lot of tools out there yet targeted to manipulating the AST, but it doesn't require special parsing/etc... – Fred the Magic Wonder Dog Jul 02 '16 at 00:02
My point is that parsing/AST production is the *easiest* part. Everything gets harder after that. The question is, do you want to reinvent all the machinery required to do transformations, or do you want use foundations where this is built in and you can work on your problem? – Ira Baxter Jul 02 '16 at 00:54

Programmatically alter Elixir Code

2 Answers2