LaTeX Property Trees

[tawheeler]

At the end of the day the PGFPlots.jl package needs to print the right property trees to a LaTeX document. Currently, many of the objects have a mix of specified properties (like Linear's xmax), and unspecified properties which can be set using style.

Some types of plots, like bar blots, are not different plot types but differ instead merely by key parameters---the presence of "ybar" for instance. Often times we want to check whether a property has been set, or set a certain property, and the fact that fields like style are not parsed makes this a little difficult.

One way to handle this is to have each plot type actually be a LaTeX property tree, kind of like JSON.

[tawheeler]

From @mykelk:

One thing that I was thinking of doing was allowing varargs and then converting those key-value pairs. I turns out that Julia's version of varargs even supports non-assignments, like "black" when plotting.

[KristofferC]

The problem with varargs is that they don't support keys containing multiple words. Of course a convention could be used so that perhaps an underline is converted to a space, however "nested" options become problematic.

Regarding property trees, here is an example I tried:

stringify(io::IO, s) = print(io::IO, s)

function stringify(io::IO, opts::Vector)
    for opt in opts
        stringify(io, opt)
        println(io, ",")
    end
end

function stringify(io::IO, p::Pair)
    print(io, first(p), " = {")
    stringify(io, last(p))
    print(io, "}")
end

Example:

stringify(STDOUT, "legend style" => ["at" => (0.5,-0.15),
                                     "anchor" => "north",
                                     "legend columns" => -1]

legend style = {at = {(0.5, -0.15)},
anchor = {north},
legend columns = {-1},
}

stringify(STDOUT, "symbolic x coords" => ["excellent", "good", "neutral"])

symbolic x coords = {excellent,
good,
neutral,
}

etc

[mykelk]

I like this idea. I'm interested in knowing @tawheeler's thoughts.

[KristofferC]

My previous suggestion was not so good because it makes it hard to change properties after the Axis is created (due to it being stored as a raw String). It is better to store it in a Dict and only do the conversion to string when exporting to a latex file.

For example:

immutable Axis
    options::Dict
end

Axis(args::Vararg{Union{String, Pair{String, T} where T}}) = Axis(dictify(args))
Base.getindex(a::Axis, s::String) = a.options[s]
Base.setindex!(a::Axis, s::String, v) = a.options[s] = v
Base.delete!(a::Axis, s::String) = delete!(a.options, s)

function dictify(args)
    d = Dict{String, Any}()
    for arg in args
        accum_opt!(d, arg)
    end
    return d
end

accum_opt!(d::Dict, opt::String) = d[opt] = nothing
accum_opt!(d::Dict, opt::Pair) = d[first(opt)] = valuify(last(opt))

valuify(x) = x
valuify(opts::Vector) = dictify(opts)


function stringify(io::IO, d::Dict)
    for (k, v) in d
        print(io, k)
        if v != nothing
            print(io, " = {")
            stringify(io, v)
            print(io, "}")
        end
        print(io, ", ")
    end
end

stringify(io::IO, s) = print(io::IO, s)

Usage:

# Creating axis
a = Axis("blue", 
     "xlabel" => "x", 
     "legend style" => ["at" => (0.5,-0.15),
             "anchor" => "north",
             "legend columns" => -1]
     )

# getindex
a["xlabel"]
# Can nest
a["legend style"]["at"]
# setindex!
a["legend style"]["anchor"] = "south"

# Make it into an option string
stringify(STDOUT, a.options)
# prints:
legend style = {anchor = {south}, at = {(0.5, -0.15)}, legend columns = {-1}, }, blue, xlabel = {x},

This also makes it easy to merge in new options, create themes etc.

[tawheeler]

I like the Dict idea much more than the original string concept. Using native julia types would also be more natural to package users.
Would you consider forking the repo and making a proof of concept?

[KristofferC]

Some other thoughts. It is quite unfortunate that there is no short form Dict syntax in Julia. The whole business with dictify up there is just because it is more convenient to use [] to denote a new block instead of Dict(). One of the problem is that some options in PGFPlots look like key => 1000, 800, 600 and they would be nice to write as key => [1000, 800, 600] but that will with the function above get "lowered" into key => {{1000}, {800}, {600}}. It is possible to use tuples but right now I thought of using tuples for the syntax key => {a}{b} (written in Julia as "key" => (a, b)). Could possibly use a unicode character to take the place of Dict. I also noticed that some keys are not strings so the code above might be a bit strictly typed.

I tinkered a bit with creating my own, more direct pgfplots syntax package this weekend and implemented some of the figures given in the PGFPlot manual.

http://nbviewer.jupyter.org/github/KristofferC/PGFPlotsXExamples/blob/master/examples/Chapter3.ipynb
http://nbviewer.jupyter.org/github/KristofferC/PGFPlotsXExamples/blob/master/examples/Chapter4.3.ipynb
http://nbviewer.jupyter.org/github/KristofferC/PGFPlotsXExamples/blob/master/examples/Chapter4.5.ipynb
http://nbviewer.jupyter.org/github/KristofferC/PGFPlotsXExamples/blob/master/examples/Chapter4.7.ipynb

[mykelk]

Oh, your notebook examples are quite nice. It is definitely a more direct translation. Maybe a direct translation is good, but perhaps with the existing PGFPlots.jl API wrapping it? In other words, can we wrap what you have so that the PGFPlots.jl notebook runs as is? There is some tricky stuff with colormaps and the handling of images.

[KristofferC]

I just noticed that options are order dependent which means that one needs an OrderedDict instead of a standard Dict...

[tawheeler]

I am really liking how this looks.
The unicode character alias can definitely be done. Maybe as const Ð = OrderedDict{String,Any} or something similar.

[KristofferC]

I found some macro code that originally was intended for easier syntax when writing JSON which I tweaked a bit. With that it is possible to write things like:

@pgf {
    "blue",
    "xlabel" = "x",
    "ylabel" = "y",
    "axis background/.style" = {
        "shade",
        "top color" = "gray",
        "bottom color" = "white"
    }
}

and get:

julia> d["xlabel"]
"x"

julia> d["axis background/.style"]["top color"]
"gray"

This syntax is quite similar to the PGFPlots syntax. You can also leave out the string quotations from the keys as long as they don't contain spaces or special characters. It could be possible to have a convention to use _ as a word separator and insert spaces in the macro but for the keys that include special characters like/.I think a string is needed. Using the macro frees up [] to actually be used as PDFPlots list values as in key = 1000, 800, 600.

The macro code can be found at https://gist.github.com/KristofferC/5e7f40eecb605c72251a109d7a6122ec

[mykelk]

I'd like to avoid the unicode. I like what you suggest @KristofferC.

I wonder... is there a huge advantage of this for the user of PGFPlots over just putting all those options into one set of quotes as done right now with the style option? If it is just to make things easier for the programmers of the internals of PGFPlots.jl, then we can take the style string and then turn it into a dictionary internally. I'm not advocating either way at this point, but I'm interested in knowing your thoughts.

[mykelk]

I'd like to avoid the unicode. I like what you suggest @KristofferC.

I wonder... is there a huge advantage of this for the user of PGFPlots over just putting all those options into one set of quotes as done right now with the style option? If it is just to make things easier for the programmers of the internals of PGFPlots.jl, then we can take the style string and then turn it into a dictionary internally. I'm not advocating either way at this point, but I'm interested in knowing your thoughts.

[mykelk]

I just spoke with @tawheeler --- what if we have PGFPlotsX.jl (or something like that) that supports the direct pgfplots generation. Then, we can refactor PGFPlots.jl to use PGFPlotsX.jl as the backend. That way, we don't have to break anyone's code, preserve the ease of use with PGFPlots.jl, preserve the Plots.jl interface, etc.

[tawheeler]

It should be possible to create a macro that directly supports PGFPlots syntax, spaces and all.

[KristofferC]

I wonder... is there a huge advantage of this for the user of PGFPlots over just putting all those options into one set of quotes as done right now with the style option?

I definitely think so. A few reasons:

• Parsing an option string sounds tedious
• It allows better syntax highlighting, separating keys and values
• Using real julia values, we can dispatch on their types to provide custom "stringifiers". With a pure string you are only left with interpolation which will in general not work well.

It should be possible to create a macro that directly supports PGFPlots syntax, spaces and all.

I am sceptic because it still has to be valid Julia syntax which is what macros work on.

julia> :(foo bar = 3)
ERROR: syntax: missing comma or ) in argument list

[mykelk]

Ah, I think you might be right on the spaces issue.

I think you make a convincing argument for this. I'm a fan.

[tawheeler]

I see, it has to support the Expr syntax so that Julia can build the AST.

[KristofferC]

I made the macro a bit better so you can just annotate a whole chain of functions and the macro will traverse through all the arguments and replace everything with { ... } into the dict style. Also, I made it so that keys with underline are exported as spaces. An example:

I must say that I like this a lot.

[mykelk]

This is beautiful!

[KristofferC]

Another thing that is pretty cool is that you can macro annotate a whole block like:

[KristofferC]

For an example of what I meant with dispatch. Below I define how how an RGB color from the Colors-package should be written when used in an option by overloading the print_opt function (bad name I know :P). We can then use a colorant directly as a key for the color option:

[mykelk]

This is very nice!

[KristofferC]

Not sure if you guys care but with exception of the image stuff here I think I am almost at feature parity now https://github.com/KristofferC/PGFPlotsXExamples/blob/master/examples/PGFPlots.jl.ipynb.

[tawheeler]

That is fantastic! I really like the syntax. It is nice that the pgf macro can encapsulate GroupPlot code too.

[mykelk]

Oh, this is very nice. Is the idea to keep the colormap stuff in PGFPlots.jl or to have it in PGFPlotsX.jl (or whatever it is called)? I think PGFPlots.jl building on top of this would greatly simplify the code within PGFPlots.jl.

[KristofferC]

Yeah, I think it makes sense to have one "low level" interface and then a more high level, user friendly interface built on top of that.

[KristofferC]

In case you are interested, I started a bit on documentation: https://kristofferc.github.io/PGFPlotsX.jl/stable/index.html

[mykelk]

Cool! Should it eventually be brought into the https://github.com/JuliaPlots organization?

[KristofferC]

Perhaps, but I think that is maybe mostly for the Plots.jl ecosystem? It even has the subtitle "Data visualization in Julia using Plots.jl"

[mykelk]

Oh, good point!

[KristofferC]

Another idea is instead of building PGFPlots.jl on top of PGFPlotsX.jl, the two packages could just be merged. It feels a bit overkill to have two packages for something as specific as PGFPlots-plotting in Julia. This would keep documentation and tests in one place which I think would be more user friendly.

[mykelk]

I'm open to that. (Great meeting you yesterday!)

[KristofferC]

Great to meet you too. Your talk was very interesting; thanks for keeping us safe while we are flying :)