Generate dependency graph for all objects in application container

[davydovanton]

I created a simple proof of concept for generation graph of all dependencies in container: link to gist file.

For this I used "custom" auto_inject register for creating included_keys class method with all dependencies.

After that I detect all classes with this method and generate "AST" with associations between classes.

With this changes we can detect unused dependencies in container, build GUI graph for easily understand system (like UML diagrams) and ect.

We can start build something like this for dry-system, WDYT? I can create a simple plan with small steps for this feature if you want.

[davydovanton]

After discussion with @solnic we decided to create a graph of all registered dependencies in dry-system. Also, we want to provide public API for working with this graph. It’ll allow using dry-system-graph for everyone.

Our strategic product goals now: build GUI graph for dependencies, detect unused keys and calculate heat map.

Implementation

We want to use dry-events and dry-monitoring for sending events that something was registered in the container. After that, create a new library for detecting it and detecting what and when was called. And build a graph depend on these events.

After that, we want to create plugins for GUI, unused keys and etc.

Schematically it can look like this. In dry-event we subscribe on something that happen in dry-system (registered, joined and called):

After that we can build a directed graph with weight for edges:

Current state and TODO

dry-events and monitoring

• Understand how to work with dry-events and update (or make documentation);
• Have a call with @mensfeld do understand what our next steps in dry-monitoring library;

dry-system dependency graph

• Update #register method for broadcasting events;
• Update injectable object for dry-system for broadcasting events on dependency injection;
  • main issue - how to understand what was injected and target for this;
• Describe dry-system monitoring/event mechanism in documentation;
• Move allow logic to dry-system (instead auto_inject);
• Create a dry-system-graph library and subscribe to dry-system notifications for generation graph of dependencies;
• Create documentation for dry-system-graph project and create a simple example of how to build own plugins;
• Make a plugin (based on the pure rack) for show dependency graph on a web page;

[timriley]

This looks like an exciting development, @davydovanton! Mind if I ask a few questions? I'm sure many of these might've been clear to those of you chatting in person, but it might help for people like me, following from afar and perhaps interested in helping.

(1) Firstly, is the goal of this to be able to get the dependency graph simply from booting the system? Or will it need the system to actually be run and exercised in order to generate the graph? (i.e. have actual web requests run over it, in the case of a web app)

(2) Would you mind explaining a bit more what you meant here?

Move allow logic to dry-system (instead auto_inject);

Thanks! I'll be following this closely and will try to help where I can!

[davydovanton]

@timriley thanks for questions! Really love it 👏

(1) I hope we will able to detect nodes/edges after call register method and include Inject[...] line. I want to use running application only for optional detecting weight across nodes (dependencies).

(2) Forgot to update comment and link this PR - dry-rb/dry-auto_inject#59

[davydovanton]

The current state of the feature:

We merged notifications to dry-system #113 and create a simple WIP project https://github.com/dry-rb/dry-system-dependency_graph. And now working on building and working with the dependency graph.

Generated dependency graph for repository example:
https://github.com/dry-rb/dry-system-dependency_graph/blob/master/examples/standalone/dependency_graph.png

[timriley]

Since the initial dependency_graph plugin is already implemented, I'm closing this issue.