internal references in "draws" parameter does not work

[yamins81]

The follow genson expression doesn't do what I thought it should:

{"parameter0" : choice([10,20]), "parameter1": gaussian(0, 1, draws=this.parameter0) }

e.g. instead of given either 10 or 20 draws from the gaussian, it only gives 1.

[yamins81]

Actually, what I really want to be able to do is something like:

{"parameter0" : choice([10,20]), "parameter1": [gaussian(0, 1) for ind in range(this.parameter0)] }

Is there some valid way to do this sort of thing in genson?

[yamins81]

OK, I think I have solved it. Basically, by implementing ParameterGenerator.genson_eval like this:

def __genson_eval__(self, context):
    self.draws = resolve(self.draws,context) 

and calling ParameterGenerator.genson_eval first inside the genson_eval methods of relevant subclasses, this sort of expression will do what I want:

{"parameter0" : choice([10,20],draws=1), "parameter1": [gaussian(0, 1,draws=this.parameter0)] }

Dave, is this a "recommended" way of solving this kind of problem?

[yamins81]

Pardon me. The above addition does make genson responsive to draws=this.parameter0, but I still can't figure out a syntax-allowable way to get variable-length lists of draws. E.g. see the comment two above. Any ideas?

[yamins81]

Aha! Ok, I think I see what I should be doing. I've added a size parameter to GaussianRandomGenerator.

Upon thinking about it, I think it never really makes sense to have "draws" by set by internal reference.

[npinto]

Wait wait, you may have solved your issue but your use case still points to a bug right?

{"parameter0" : choice([10,20]), "parameter1": gaussian(0, 1, draws=this.parameter0) } should output {"parameter0" : 10, "parameter1": eval("gaussian(0, 1, draws=10)") } or {"parameter0" : 20, "parameter1": eval("gaussian(0, 1, draws=20)") }

and it does not:

In [1]: import genson

In [2]: genson.loads('{"parameter0" : choice([10,20]), "parameter1": gaussian(0, 1, draws=this.parameter0) }')
Out[2]: <genson.JSONGenerator instance at 0x102b081b8>

In [3]: [e for e in genson.loads('{"parameter0" : choice([10,20]), "parameter1": gaussian(0, 1, draws=this.parameter0) }')]
Out[3]: [{'parameter0': 10, 'parameter1': 0.10782061661484783}]

In [4]: [e for e in genson.loads('{"parameter0" : choice([10,20]), "parameter1": gaussian(0, 1, draws=this.parameter0) }')]
Out[4]: [{'parameter0': 20, 'parameter1': -0.4968957047785028}]

In [5]: [e for e in genson.loads('{"parameter0" : choice([10,20]), "parameter1": gaussian(0, 1, draws=this.parameter0) }')]
Out[5]: [{'parameter0': 20, 'parameter1': 1.4981330113708884}]

[yamins81]

Well, to get the "correct" response on the things you wrote, you'd have to make the small change by implementing ParameterGenerator.genson_eval as I describe above ...

... but then, I realized that it's not really what I wanted anyway. And the correct thing is not to use the 'draws' variable at all, but instead to implement a size variable in the base distributions.

[npinto]

Since you got the fix, it would be nice to get a PR with the bug being exposed and fixed. Thanks in advance.

[yamins81]

Well, so I didn't submit it as a PR because it wasn't obvious if it EVER really makes sense to have "draws" bye set by internal reference. Not only isn't it "quite" the right solution to my problem, it seems like it might be a kind of "mistaken concept" to me, one that should be prohibited by the API.

[npinto]

It may not make sense for your application and you may have found a workaround but this is still a bug. function kwargs should be able to resolve correctly or raise an exception.

[npinto]

Example:

In [1]: import genson
In [2]: r =  [e for e in genson.loads('{"rseed": choice([24, 42]), "param0": gaussian(0, 1, draws=1, random_seed=this.rseed)}')]
ERROR: An unexpected error occurred while tokenizing input
The following traceback may be corrupted or invalid
The error message is: ('EOF in multi-line statement', (187, 0))
...
ValueError: object of too small depth for desired array

[yamins81]

I'm not 100% sure I understand. Are you saying:

1. That I should submit a PR that raises an exception if one tries to put an internal reference for "draws"? If so, I can do this.

or
2) You want genson to have the behavior that you suggested it have earlier? I essentially know how to make it have this, but even though at first I thought this was the right behavior, now I kind of am not so sure it makes sense to me, and maybe would argument for option 1).

Dave, do you have an opinion about this?

[yamins81]

Whoops, I should I have put @davidcox on this ...

Also: @npinto, I am getting a PR ready for the "sizes arguments in distributions" -- that actually solves my problem, and I think it is sort of orthogonal to the question we've been discussing here. So I can ask for PR independently on that one, regardless of what we decide about this current is. (you can see the feature/distribution_sizes branch in my fork)

[npinto]

Nice, size would be awesome to have!

Regarding resolving kwargs, I think we should have this work as expected (instead of raising an exception), in case you want to "link" genson function kwargs in a complicated genson expression. What do you think?