Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible discussion topics: Default colorbar limits and misfit normalization #259

Open
rmodrak opened this issue Mar 22, 2024 · 9 comments

Comments

@rmodrak
Copy link
Member

rmodrak commented Mar 22, 2024

Two separate but related issues here

@rmodrak
Copy link
Member Author

rmodrak commented Mar 22, 2024

First, Should we introduce a colorbar saturation option? If so, what should be the default? See figure below for one possible approach.

Screen Shot 2024-03-22 at 12 04 23 AM

@rmodrak
Copy link
Member Author

rmodrak commented Mar 22, 2024

Second, should we turn the following normalization on by default?

https://github.com/rmodrak/mtuq/blob/9c53730e65ef83f4deb18a1a55efd642749a98ee/mtuq/misfit/waveform/level0.py#L25

With this turned on, plotting variance reduction could become as simple as

from graphics import plot_misfit_lune
plot_misfit_lune(filename, 1. - normalized_misfit_dataarray)

@thurinj
Copy link
Member

thurinj commented Mar 24, 2024

Hi @rmodrak,

Thanks for bringing this up. I don't know what to do regarding the clipping value, as it would be very case-dependent. Some examples don't even need any clipping to be readable. Would an alternative scale make sense?
Another approach I like is to clip at a certain percentile of data: Given all the values to be displayed in the misfit map, clip the values such that the map explains n% of the map data.

As for the normalization, I would advocate for turning it on by default. It makes more sense to have all misfits in terms of data norm, and it would also help users understand data weight a bit more. Our canonical example is primarily dominated by the surface waves (at about 1:10 ?), and it isn't obvious based on absolute misfit alone. And I also agree that it would make other metrics easier to access.

@rmodrak
Copy link
Member Author

rmodrak commented Mar 25, 2024

Good points, thanks. Agreed, the current plotting works well in most cases. To add some context, there is some specific motivation here based on sponsor feedback.

The motivation has to do with historical events with sparse data coverage and weakly constrained source type. In some sense, the current plotting does its job for such events by conveying a weakly constrained solution. However, we have gotten some feedback that it can still sometimes be useful to see details of the misfit contour orientation, even for these weakly constrained events.

@rmodrak
Copy link
Member Author

rmodrak commented Mar 25, 2024

Given the above, let me go ahead and add a colorbar clipping option, but keep it turned off by default for now.

Agreed that "clipping" is a better description of the proposed approach than "saturation"-- let me make this change in my fork.

Jonas had the same idea about clipping based on percentile. However, one issue is that the percentile distribution depends strongly on how fine the sampling is at the poles relative to the center of the lune? For now, let me implement clipping based on extremal misfit values only-- we could discuss further over Zoom, if there is interest.

Let me go ahead and make theses changes via a pull request tomorrow, if that's alright.

@thurinj
Copy link
Member

thurinj commented Mar 25, 2024

Thanks Ryan.

The static images in the docs will also probably have to be re-generated as the solutions for the 2009 examples are going to change with misfit normalization.

As for the percentile clipping: Given all the values are mapped out before any clipping is applied, does the overall spatial repartition of data on the lune play a strong role if we do the percentile mapping? I reckon it would just depend on the actual range of values displayed.

I have mostly used it when mapping CMA-ES samples on the lune, where the stochastic nature of the sampled values (especially at the beginning) generates big gaps in the misfit values displayed. With percentile clipping, I can still see what's going on around the minima without the map being dominated by values orders of magnitude higher than the minimum (but that might also be why it is so useful in this case).

Thanks!

@rmodrak
Copy link
Member Author

rmodrak commented Mar 25, 2024

I can still see what's going on around the minima without the map being dominated by values orders of magnitude higher than the minimum

Well said, this is a good way of putting it

@rmodrak
Copy link
Member Author

rmodrak commented Mar 25, 2024

For tomorrow let me push some small changes, which will not at all affect the default figure appearance. Interested in others' feedback before making any changes to the default color limits or default misfit normalization...

@rmodrak
Copy link
Member Author

rmodrak commented Mar 27, 2024

@ jonas

To bring out more visual contrast in the misfit surface, we can reduce colorbar range using cpt_clip and the number of contours levels using cpt_steps. For example

plot_misfit_lune(filename, data_array, cpt_steps=10, cpt_clip=[0.,0.5])

For now, these new keyword arguments remain turned off by default and subject to change pending further feedback.

One thing to note is that not just plot_lune_misfit, but all the GMT plotting utilities can accept these arguments.

The detailed implementation can be followed here

def _call(shell_script, filename, data, supplemental_data=None,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants