get the variable that causes a false output on jsonlogic evaluation

[FlorianRuen]

Hey @diegoholiveira

I'm using JsonLogic in Go backend, with sometimes complex logics (lot of sums, conditional ...)
And in order to help my end users to debug in case of false result, I'm asking if there is a way to output the variable that cause the false output

For example, output just need to be a string with variable name

{
  "and":[
    { "==":[{ "var":"VariableA" }, true] },
    { ">=":[{ "var":"VariableC" }, 17179869184] },
  ]
}

If VariableA = false and VariableC = 17179869184
The jsonlogic.Apply() output can be result, variable, err := jsonlogic.Apply()
With variable = "VariableA" (or maybe an array in case of multiple var, or maybe only the first that cause false result)

Do you think, this kind of behavior can be achieve, or because of the operation of jsonlogic which works with json, this will not be possible?

If yes, if you have some tips, maybe I can achieve this, and make another PR !

[diegoholiveira]

It would be super nice and useful to have something like this, but I'm not sure how we can achieve it. I do think it would require another data structure to achieve this, one that allow the output of each step of the json logic eval.

I will think about it, but I have not idea how to do it right now. I'll be glad to accept a PR.

[FlorianRuen]

With another json logic eval, it can be achieve using something like

{
  "if": [
    { "<": [{ "var": "VariableA" }, 5] },
    "VariableA is less than 5",
    { ">": [{ "var": "VariableA" }, 10] },
    "VariableA is greater than 10",
    null
  ]
}

So the output will be a clear message, but my idea is to achieve this without changing the json logic, because, it can be really tricky when the json logic is built using an UI side (the users can create the jsonlogic)

[diegoholiveira]

I was thinking about it and I do think a good API to this problem is:

parsed := jsonlogic.New(logic).With(jsonlogic.EXPLAIN).parse(data)

// an alternative
// parsed := jsonlogic.New(logic).With(jsonlogic.TRACE).parse(data)

What do you think? @joaoandre please take a look into this thread.

[FlorianRuen]

@diegoholiveira Can be a very good way to achieve this
And the parsed variable in output, the result should be the entire jsonlogic ? or only the concerned variable ?

[joaoandre]

@diegoholiveira I do like the API you propose(I like TRACE or TRACEBACK better than EXPLAIN).

Maybe the result of parse could be a map with both the result and the traceback of apply. Something linke this:

// It would have to return an error as well to be compliance with the apply api
parsed, err := jsonlogic.New(logic).With(jsonlogic.TRACE).parse(data)
// parsed would be like
// {"result": ..., "trace": ...}

But maybe it would be too complex to combine both at the moment. I still not quite sure how this would be implemented.

[diegoholiveira]

I'm thinking about the output to be like this:

interface Output {
    Value() any
    Trace() any
    Error() error
}

func (e *Engine) Parse(data any) Output {
    // TODO
}

It's very close to your proposal (an interface instead of map), but sure, it would require a full rewrite to introduce this tracing. It could be a good moment to also apply generics and reduce the usage of reflection.

[FlorianRuen]

That suggestion can be a very good way!
Maybe the output using @joaoandre suggestion can be better, but both of them answer to the main issue

interface Output {
    Value() any
    Trace() any
}

result, err := jsonlogic.New(logic).With(jsonlogic.TRACE).parse(data)
// parsed would be like
// {"result": ..., "trace": ...}

func (e *Engine) Parse(data any) (Output, error) {
    // TODO
}

[FlorianRuen]

Hello @diegoholiveira

Is this improvement still in progress/or perhaps scheduled ?
Maybe it still requires a little more thought.

[diegoholiveira]

Hey @FlorianRuen . Unfortunately I'm not being able to deep thought about it yet. It's on my mind to do it but I'm little busy and taking care of the small issues only.

Do you have any proposal? I would be glad to hear from you because you're the first real user of this feature.

[TotalTechGeek]

Hey @FlorianRuen,

I've had some time to dwell on this a bit since this guy: json-logic/json-logic-engine#15

I also provided some similar input the other day when commenting on a .NET Project: json-everything/json-everything#818

As mentioned in the .NET post,

I'm starting to wonder if the best way to approach this is to produce a common validator extension for JSON Logic and introduce a set of helpful operators that make it more convenient to process the logic.

That way only a few new operators are needed to accommodate this, rather than a potentially complex tracing mechanism built into every implementation of JSON Logic.

---

Example of a potential validator extension (I'm using a different notation because it makes JSON Logic less of a pain to read):

validate( 
  @.OrderDate > '2024-11-17T18:30:00.000Z',
  some(@.OrderItemDetails, @.LineId < 0),
  @.OrderStatus == 4
) // "1"

Perhaps it could be paired with a few other operators, like validateSome or validateAll

validate(
  @.OrderDate > '2024-11-17T18:30:00.000Z',
  validateSome(@.OrderItemDetails, @.LineId < 0),
  @.OrderStatus == 4
) // "1.0"

If some variant of this were implemented, it'd potentially be easy to s/and/validate and s/or/validateAny

(In a moment, I might add an example implementation in JLE, and set it up to return the logic rather than the path)

---

My current goal is to try to improve JSON Logic compatibility between different ecosystems, but once that has been achieved, I think it might be nice to get the community to potentially propose some new common extensions to the spec.

The reason this isn't default behavior is because JSON Logic serves as more of an AST / Lisp-like language for sandboxing application capabilities and rules, than as a validation library.

You can certainly validate information in JSON Logic, as it's a declarative programming construct, but JSON Logic isn't implicitly a validation lib.

The use case comes up often though, so it might be worthwhile to develop something that simplifies it, without complicating the core interpreter implementations.

[TotalTechGeek]

To demonstrate what a potential set of "validator" operators could potentially look like:
https://gist.github.com/TotalTechGeek/48d31da03c1148e4360cf486515b96dc

Using the implementation in the gist, and doing some subsitutions,

// I'd normally advise against doing this, but I'm doing this to keep the cognitive overhead low.
engine.methods.and = engine.methods.validate
engine.methods.or = engine.methods.validateAny
engine.methods.some = engine.methods.validateSome
engine.methods.every = engine.methods.validateEvery

const hasIssues = engine.build({
  and: [
    { '>': [{ var: 'age' }, 18] },
    { '===': [{ var: 'name' }, 'John'] },
    {
      or: [
        { '===': [{ var: 'cool' }, true] },
        { '===': [{ var: 'interesting' }, true] }
      ]
    },
    {
      every: [
        { var: 'friends' },
        { '===': [{ var: '' }, 'Joe'] }
      ]
    }
  ]
})

console.log(JSON.stringify(hasIssues({ name: 'John', age: 19, friends: ['Joe'] })))
// =>> {"logic":[{"===":[{"var":"cool"},true]},{"===":[{"var":"interesting"},true]}],"context":{"name":"John","age":19,"friends":["Joe"]},"path":"2.*"}

console.log(JSON.stringify(hasIssues({ name: 'John', age: 19, interesting: true, friends: ['Joe'] })))
// =>> null

console.log(JSON.stringify(hasIssues({ name: 'John', age: 17, friends: ['Joe'] })))
// =>> {"logic":{">":[{"var":"age"},18]},"context":{"name":"John","age":17,"friends":["Joe"]},"path":"0"}

console.log(JSON.stringify(hasIssues({ name: 'Jane', age: 19, friends: ['Joe'] })))
// =>> {"logic":{"===":[{"var":"name"},"John"]},"context":{"name":"Jane","age":19,"friends":["Joe"]},"path":"1"}

console.log(JSON.stringify(hasIssues({ name: 'John', age: 19, cool: true, friends: ['Jane'] })))
// =>> {"logic":{"===":[{"var":""},"Joe"]},"context":"Jane","path":"3.item[0]"}

It could / should be possible to shim this functionality into various JSON Logic interpreters without complicating the root spec ;)

This specific impl is leveraging traverse: false in JLE, which is only supported in about half of the JSON Logic Interpreters I've seen; without it, this could still be implemented, but the rules would not be evaluated lazily.