Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove shape column from DataCommon csv download #365

Open
theryankelly opened this issue Dec 24, 2020 · 2 comments
Open

Remove shape column from DataCommon csv download #365

theryankelly opened this issue Dec 24, 2020 · 2 comments

Comments

@theryankelly
Copy link
Collaborator

Describe the bug
Also listed in Zoning Atlas Issue #43. The shape column is causing issues for csv export is opened in Excel. Need to remove form that export.

Steps to reproduce
Thread on how to address

Expected behavior
csv should not have the shape file included. It's expect that removing that column we will not have the line breaks and shifted rows because shape column was breaking excel character limit

Screenshots
If applicable, add screenshots to help explain your problem.
image

Additional context
Temp fix has been to change download link on Zoning Atlas to shapefile rather than DataCommon where intended. When fixed need to update zoning atlas links and about page text on where to find data.

@mzagaja
Copy link
Contributor

mzagaja commented Jan 4, 2021

Guy suggested using a function to get all the column names and remove the shape column from the CSV export. However trying this query in the code base:

-c "\\copy \(SELECT 'SELECT ' || STRING_AGG('t.' || column_name, ', ') || ' FROM #{table_name} AS t' FROM information_schema.columns WHERE table_name = '#{table_name}' AND table_schema = 'mapc' AND column_name NOT IN ('shape'))

Does not work. This is potentially due to the fact that the \copy function does not support variable interpolation:
image (1)

If this is important for users then the workaround would be to load a version of the dataset into the DataCommon with a slightly different name in the tabular schema that sufficiently indicates to the user that dataset might be more useful for tabular purposes if they are an excel user. A more robust (and longer to write) version of the feature would allow the dual uploads and then add logic to all the templates to pick the appropriate tabular v. spatial sources based on which are available. Note that if we do this then going forward the data administrator will need to remember to and update both the tabular and geospatial datasets when there are changes.

@atomtay
Copy link
Contributor

atomtay commented Jan 5, 2021

I'm a bit confused about the second proposed solution; don't we already have dual uploads (tabular CSV and spatial SHP)?

mzagaja added a commit that referenced this issue Jan 6, 2021
* Because the Zoning Atlas data did not properly display for users who chose to use Excel as their CSV viewer, we are now providing a Sharepoint link that is in XLSX format for their conveinence.
* Even more annoying was that the shapefile did not include any shapes when we exported it. So we also had to create the shapefile manually and put into sharefile.

Related to #366 and #365, but does not fix them beyond Zoning Atlas.
mzagaja added a commit that referenced this issue Jan 6, 2021
* Because the Zoning Atlas data did not properly display for users who chose to use Excel as their CSV viewer, we are now providing a Sharepoint link that is in XLSX format for their conveinence.
* Even more annoying was that the shapefile did not include any shapes when we exported it. So we also had to create the shapefile manually and put into sharefile.

Related to #366 and #365, but does not fix them beyond Zoning Atlas.
@mzagaja mzagaja removed their assignment Mar 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants