Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Save and apply jigsaw adjustments #5419

Merged
merged 17 commits into from
Sep 20, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions isis/src/base/objs/Table/Table.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ find files of those names at the top level of this repository. **/
#include "Table.h"

#include <fstream>
#include <sstream>
#include <string>

#include "Blob.h"
Expand Down Expand Up @@ -137,6 +138,60 @@ namespace Isis {
}
}

/**
* This constructor takes in a string to create a Table object.
*
* @param tableName The name o the Table to be read
* @param tableStr The table string
* @param fieldDelimiter The delimiter to separate fields with
*/
Table::Table(const QString &tableName, const std::string &tableString, const char &fieldDelimiter) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if we are reading in Jigsaw results we should be able to use the ISIS CSVReader class instead of implementing the reader in Table. I could be missing something about our CSV Reader though.

Something to try when we get back to this

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CSVReader was kind of a mess. I dont mind re-visiting trying it, originally, we wanted to just write out the blobs using with something like Table -> toBlob -> write(stringstream) -> write to HDF5. But that also was a mess. If we revisit I think trying the blob route would be best. She was able to get it work with a quick ghetto method than I was trying to de-tangle Blob writing with streams. Only spent a day on it, but I was getting a bunch of segfaults and stream errors for whatever reason.

p_name = tableName;

std::stringstream tableStream;
tableStream << tableString;

// Create fieldNames <-- refactor in progress
std::vector<std::string> tableLinesStringList;
std::string line;
while(std::getline(tableStream, line, '\n')) {
tableLinesStringList.push_back(line);
}

int numOfFieldValues = tableLinesStringList.size() - 1; // minus the header line

std::string fieldNamesLineString = tableLinesStringList.front();
std::stringstream fieldNamesStringStream;
fieldNamesStringStream << fieldNamesLineString;

std::vector<QString> fieldNames;
std::string fieldNameString;
while(std::getline(fieldNamesStringStream, fieldNameString, fieldDelimiter)) {
fieldNames.push_back(QString::fromStdString(fieldNameString));
}

// Clear error flags and set pointer back to beginning
tableStream.clear();
tableStream.seekg(0, ios::beg);

// Add records to table
std::string recordString;
int index = 0;
while(std::getline(tableStream, recordString, '\n')) {
// skip first line bc that's the header line
if (index == 0) {
index++;
continue;
}

TableRecord tableRecord(recordString, fieldDelimiter, fieldNames, numOfFieldValues);
p_record = tableRecord;
std::cout << "Adding record" << std::endl;
this->operator+=(tableRecord);
index++;
}
}


/**
* Initialize a Table from a Blob that has been read from a file.
Expand Down
1 change: 1 addition & 0 deletions isis/src/base/objs/Table/Table.h
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ namespace Isis {
Table(const QString &tableName, const QString &file,
const Pvl &fileHeader);
Table(const Table &other);
Table(const QString &tableName, const std::string &tableString, const char &fieldDelimiter);
Table &operator=(const Isis::Table &other);

~Table();
Expand Down
18 changes: 17 additions & 1 deletion isis/src/base/objs/TableRecord/TableRecord.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ find files of those names at the top level of this repository. **/
#include "TableRecord.h"

#include <iostream>
#include <sstream>
#include <string>
#include <vector>

Expand All @@ -21,6 +22,21 @@ namespace Isis {
TableRecord::TableRecord(){
}

TableRecord::TableRecord(std::string tableRecordStr, char fieldDelimiter,
std::vector<QString> fieldNames, int numOfFieldValues) {
std::stringstream tableRecordStream;
tableRecordStream << tableRecordStr;

std::string fieldStr;
int i = 0;
while(std::getline(tableRecordStream, fieldStr, fieldDelimiter)) {
TableField tableField(fieldNames[i], TableField::Double);
tableField = std::stod(fieldStr); // convert string to double
p_fields.push_back(tableField);
i++;
}
}

//! Destroys the TableRecord object
TableRecord::~TableRecord() {
}
Expand Down Expand Up @@ -155,7 +171,7 @@ namespace Isis {
Isis::TableField &field = p_fields[f];
field = (void *)&buf[sbyte];
sbyte += field.bytes();
}
}
}

/**
Expand Down
2 changes: 2 additions & 0 deletions isis/src/base/objs/TableRecord/TableRecord.h
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ namespace Isis {
class TableRecord {
public:
TableRecord();
TableRecord(std::string tableRecordStr, char fieldDelimiter,
std::vector<QString> fieldNames, int numOfFieldValues);
~TableRecord();


Expand Down
85 changes: 73 additions & 12 deletions isis/src/control/apps/jigsaw/jigsaw.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,19 @@ find files of those names at the top level of this repository. **/
/* SPDX-License-Identifier: CC0-1.0 */

#include <iostream>
#include <highfive/H5File.hpp>
#include <highfive/H5DataType.hpp>
#include <highfive/H5DataSet.hpp>
#include <highfive/H5Group.hpp>
#include <vector>

#include <QDir>
#include <QList>
#include <QObject>
#include <QSharedPointer>
#include <QString>


#include "Blob.h"
#include "BundleAdjust.h"
#include "BundleObservationSolveSettings.h"
Expand All @@ -35,6 +41,8 @@ find files of those names at the top level of this repository. **/
#include "jigsaw.h"

using namespace std;
using namespace HighFive;


namespace Isis {

Expand Down Expand Up @@ -130,24 +138,25 @@ namespace Isis {
bundleSolution->outputResiduals();
}

// write lidar csv output file
if (ui.GetBoolean("LIDAR_CSV")) {
bundleSolution->outputLidarCSV();
}
// write lidar csv output file
if (ui.GetBoolean("LIDAR_CSV")) {
bundleSolution->outputLidarCSV();
}

// write updated control net
bundleAdjustment->controlNet()->Write(ui.GetFileName("ONET"));

// write updated lidar data file
if (ui.WasEntered("LIDARDATA")) {
if (ui.GetString("OLIDARFORMAT") == "JSON") {
bundleAdjustment->lidarData()->write(ui.GetFileName("OLIDARDATA"),LidarData::Format::Json);
}
else {
bundleAdjustment->lidarData()->write(ui.GetFileName("OLIDARDATA"),LidarData::Format::Binary);
// write updated lidar data file
if (ui.WasEntered("LIDARDATA")) {
if (ui.GetString("OLIDARFORMAT") == "JSON") {
bundleAdjustment->lidarData()->write(ui.GetFileName("OLIDARDATA"),LidarData::Format::Json);
}
else {
bundleAdjustment->lidarData()->write(ui.GetFileName("OLIDARDATA"),LidarData::Format::Binary);
}
}
}
PvlGroup gp("JigsawResults");
QString adjustmentOutputFilename;
// Update the cube pointing if requested but ONLY if bundle has converged
if (ui.GetBoolean("UPDATE") ) {
if ( !bundleAdjustment->isConverged() ) {
Expand All @@ -156,6 +165,19 @@ namespace Isis {
throw IException(IException::Unknown, msg, _FILEINFO_);
}
else {
if (ui.GetFileName("ADJUSTMENT_OUTPUT") == NULL) {
gp += PvlKeyword("Status","If UPDATE=True then must specify bundle adjustment values to update cube files.");
QString msg = "ADJUSTMENT_OUTPUT value is missing";
throw IException(IException::Unknown, msg, _FILEINFO_);
}
adjustmentOutputFilename = ui.GetFileName("ADJUSTMENT_OUTPUT");
std::cout << "AdjustmentOutputFilename=" << adjustmentOutputFilename << std::endl;
// Initialize dataset for adjustment tables
File file(adjustmentOutputFilename.toStdString(), File::Truncate);


std::vector<std::string> adjustment_list;

for (int i = 0; i < bundleAdjustment->numberOfImages(); i++) {
Process p;
CubeAttributeInput inAtt;
Expand All @@ -176,7 +198,9 @@ namespace Isis {

// Update the image parameters
QString jigComment = "Jigged = " + Isis::iTime::CurrentLocalTime();

if (c->hasBlob("CSMState", "String")) {
std::cout << "Cube has CSMState blob" << std::endl;
Blob csmStateBlob("CSMState", "String");
// Read the BLOB from the cube to propagate things like the model
// and plugin name
Expand All @@ -193,9 +217,46 @@ namespace Isis {
spvector.Label().addComment(jigComment);
c->write(cmatrix);
c->write(spvector);

QString serialNumber = bundleAdjustment->serialNumberList()->serialNumber(i);
QString cmatrixName = cmatrix.Name();
QString spvectorName = spvector.Name();
std::cout << "serialNumber=" << serialNumber.toStdString() << std::endl;

std::string cmatrixKey = serialNumber.toStdString() + "/" + cmatrixName.toStdString();
std::string spvectorKey = serialNumber.toStdString() + "/" + spvectorName.toStdString();

if (adjustmentOutputFilename != NULL) {

std::string cmatrixTableStr = Table::toString(cmatrix).toStdString();
DataSet dataset = file.createDataSet<std::string>(cmatrixKey, cmatrixTableStr);
std::string spvectorTableStr = Table::toString(spvector).toStdString();
dataset = file.createDataSet<std::string>(spvectorKey, spvectorTableStr);

// TODO: Add metadata to h5 file
PvlKeyword timeDependentFrames = c->label()->findObject(cmatrixName).findKeyword("TimeDependentFrames");
}
if (ui.WasEntered("ADJUSTMENT_INPUT")) {
QString adjustmentInputFilename = ui.GetFileName("ADJUSTMENT_INPUT");
std::cout << "AdjustmentInputFilename=" << adjustmentInputFilename << std::endl;
File fileRead(adjustmentInputFilename.toStdString(), File::ReadOnly);
DataSet datasetRead = fileRead.getDataSet(cmatrixKey);
std::vector<std::string> cmatrixData;
datasetRead.read(cmatrixData);
for (int i = 0; i < cmatrixData.size(); i++) {
Table cmatrixTable(QString::fromStdString(cmatrixKey), cmatrixData[i], ',');
// WIP <---
}
}

// flag = output to this file
}
p.WriteHistory(*c);
}
// std::cout << "Writing out dataset" << std::endl;
// dataset.write(adjustment_list);
// std::cout <<"After writing to dataset" << std::endl;

gp += PvlKeyword("Status", "Camera pointing updated");
}
}
Expand Down
28 changes: 28 additions & 0 deletions isis/src/control/apps/jigsaw/jigsaw.xml
Original file line number Diff line number Diff line change
Expand Up @@ -680,6 +680,34 @@
</filter>
</parameter>

<parameter name="ADJUSTMENT_OUTPUT">
<type>filename</type>
<fileMode>input</fileMode>
<brief>
Outputs adjustment values
</brief>
<description>
"Outputs adjustment values"
</description>
<filter>
*.h5
</filter>
</parameter>

<parameter name="ADJUSTMENT_INPUT">
<type>filename</type>
<fileMode>input</fileMode>
<brief>
Reads in adjustment values
</brief>
<description>
"Reads in adjustment values"
</description>
<filter>
*.h5
</filter>
</parameter>

<parameter name="HELDLIST">
<type>filename</type>
<internalDefault>none</internalDefault>
Expand Down
30 changes: 30 additions & 0 deletions isis/tests/TableTests.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -295,3 +295,33 @@ TEST(TableTests, Clear) {

EXPECT_EQ(t.Records(), 0);
}

TEST(TableTests, FromString) {
std::string tableStr = "J2000Ang1,J2000Ang2,J2000Ang3\n"
"-1.0261086365746,1.3843980236775,0.97666760713915\n"
"-0.026127047776247,0.034245411189199,0.0052635095732964\n"
"-0.005717949450684,-0.0039014897927048,2.3750859084069e-05\n"
"260093852.48957,46.12915199995,2.0\n";

std::cout << "tableStr=" << tableStr << std::endl;

QString tableName = QString::fromStdString("TestTableName");

std::stringstream tableStrStream;
tableStrStream << tableStr;

Table table(tableName, tableStr, ',');
std::cout << "Created table with table string" << std::endl;
std::cout << "Table name=" << table.Name().toStdString() << std::endl;
std::cout << "Table recordFields=" << static_cast<int>(table.RecordFields()) << std::endl;
std::cout << "Table recordSize=" << static_cast<int>(table.RecordSize()) << std::endl;

for (int i = 0; i < table.Records(); i++) {
std::cout << "Table[" << i << "] record=" << TableRecord::toString(table[i]).toStdString() << std::endl;
}

QString tableToString = Table::toString(table);
std::cout << "tableToString=" << tableToString.toStdString() << std::endl;

EXPECT_EQ(tableStr, tableToString.toStdString());
}