Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IO/config: parse from XML/JSON formatted string #309

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 110 additions & 0 deletions src/IO/configuration.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@

#include "configuration.hpp"
#include "configuration_XML.hpp"
#include "compiler.hpp"

#if HAS_RAPIDJSON_LIB
#include "configuration_JSON.hpp"
Expand Down Expand Up @@ -360,4 +361,113 @@ namespace config {

}

/*!
\class ConfigStringParser
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this new class really needed? Can we just add an overload to the read/write functions of ConfigParser, this overloads will take in input the string and the format?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Introducing a class of its own, without overhead of filesystem checks, version control, propagation to static utilities and whatsover was faster and simpler. As I mention earlier, introducing rapidXML instead of libxml2 can straightforward the I/O part a lot.

\ingroup Configuration
\brief Configuration parser to c++ string

This class implements a configuration parser to absorb/flush directly on
c++ string buffers.
*/


/*!
Construct a new parser.

\param XMLorJSON false to activate XML format, true for JSON format if supported,
otherwise fallback automatically to XML.
*/
ConfigStringParser::ConfigStringParser(bool XMLorJSON)
roccoarpa marked this conversation as resolved.
Show resolved Hide resolved
: Config(false)
{
#if HAS_RAPIDJSON_LIB
m_xmlOrJson = XMLorJSON;
#else
BITPIT_UNUSED(XMLorJSON);
//revert to default XML
m_xmlOrJson = false;
#endif
}

/*!
Construct a new parser, activating multiSections support.

\param XMLorJSON false to activate XML format, true for JSON format if supported,
otherwise fallback automatically to XML.
\param multiSections if set to true the configuration parser will allow
multiple sections with the same name
*/
ConfigStringParser::ConfigStringParser(bool XMLorJSON, bool multiSections)
: Config(multiSections)
{
#if HAS_RAPIDJSON_LIB
m_xmlOrJson = XMLorJSON;
#else
BITPIT_UNUSED(XMLorJSON);
//revert to default XML
m_xmlOrJson = false;
#endif
}

/*!
Return a bool to identify the format targeted by the class to parse/write the
buffer string. 0 is XML, 1 is JSON

\return boolean identifying the target format
*/
bool ConfigStringParser::getFormat()
{
return m_xmlOrJson;
}

/*!
Absorb the targeted buffer string formatted as specified in class construction
(XML or JSON).

\param source target buffer string to read
\param append controls if the buffer string contents will be appended to the
current configuration or if the current configuration will be overwritten
by them.
*/
void ConfigStringParser::read(const std::string &source, bool append)
{
// Processing not-append requests
if (!append) {
Config::clear();
}

if(m_xmlOrJson){
#if HAS_RAPIDJSON_LIB
config::JSON::readBufferConfiguration(source, this);
#endif
}else{
config::XML::readBufferConfiguration(source, this);
}
}

/*!
Flush the configuration to a buffer string, with the format specified
in class construction (XML or JSON).

\param source target buffer string to write on
\param append controls if the configuration contents will be appended to the
target string or if the string will be overwritten by them.
*/
void ConfigStringParser::write(std::string &source, bool append) const
{
// Processing not-append requests
if (!append) {
source.clear();
}

if(m_xmlOrJson){
#if HAS_RAPIDJSON_LIB
config::JSON::writeBufferConfiguration(source, this);
#endif
}else{
config::XML::writeBufferConfiguration(source, this, "root");
}

}

}
17 changes: 17 additions & 0 deletions src/IO/configuration.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,23 @@ namespace config {

};

// Configuration parsing from/ writing to string
class ConfigStringParser : public Config
{

public:
ConfigStringParser(bool XMLorJSON = false);
ConfigStringParser(bool XMLorJSON, bool multiSections);

bool getFormat();

void read(const std::string &source, bool append = true);
void write(std::string & source, bool append = true) const;

private:
bool m_xmlOrJson;
};

}

#endif
73 changes: 73 additions & 0 deletions src/IO/configuration_JSON.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,42 @@ void readConfiguration(const std::string &filename, Config *rootConfig)
readNode("", jsonRoot, rootConfig);
}

/*!
Reading a json dictionary from plain c++ string and fill a bitpit::Config tree
accordingly.

String Encoding is always considered of UTF-8 type .

\param[in] source string with JSON contents
\param[in,out] rootConfig is a pointer to Config tree that will be used
to store the data read from the string
*/
void readBufferConfiguration(const std::string &source, Config *rootConfig)
{
if (!rootConfig) {
throw std::runtime_error("JSON::readDoc Null Config tree structure passed");
}

// Open a UTF-8 compliant rapidjson::StringStream
rapidjson::StringStream str(source.c_str());
// Parse stream directly
rapidjson::Document jsonRoot;
jsonRoot.ParseStream(str);

// Handle parse errors
if (jsonRoot.HasParseError()) {
std::string message = "JSON:readBufferConfiguration error of type : ";
message += std::string(rapidjson::GetParseError_En(jsonRoot.GetParseError()));
throw std::runtime_error(message.c_str());
}

// Root should be an object, bitpit doens't support root arrays.
assert(jsonRoot.IsObject() && "JSON:readBufferConfiguration parsed document root is not an object");

// Fill the configuration
readNode("", jsonRoot, rootConfig);
}

/*!
Read recursively a json object content and fill accordingly the Config tree
branch.
Expand Down Expand Up @@ -246,6 +282,43 @@ void writeConfiguration(const std::string &filename, const Config *rootConfig, b
std::fclose(fp);
}

/*!
Write a bitpit::Config tree contents to json formatted c++ string.

NOTE: JSON root document object does not have a key name unlike XML.

Tree contents will be written to a plain c++ string, with UTF-8 standard encoding, and
appended to any previous content of the source string.

\param[in] source string to write JSON contents.
\param[in] rootConfig pointer to the Config tree to be written on target string
*/
void writeBufferConfiguration(std::string &source, const Config *rootConfig)
roccoarpa marked this conversation as resolved.
Show resolved Hide resolved
{
if (!rootConfig) {
throw std::runtime_error("JSON::writeConfiguration Null Config tree structure passed");
}

// DOM Document is GenericDocument<UTF8<>>
rapidjson::Document jsonRoot;

// Get the allocator
rapidjson::Document::AllocatorType &allocator = jsonRoot.GetAllocator();

// Create a root node and recursively fill it
jsonRoot.SetObject();
writeNode(rootConfig, jsonRoot, allocator);

// Write on a buffer with initial capacity 1024
rapidjson::StringBuffer buffer(0, 1024);

// Use single-ultracompact UTF-8 JSON format
rapidjson::Writer<rapidjson::StringBuffer> writer(buffer);
jsonRoot.Accept(writer);

source += std::string(buffer.GetString());
}

/*!
Write recursively Config tree contents to JSON objects

Expand Down
2 changes: 2 additions & 0 deletions src/IO/configuration_JSON.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,11 @@ namespace config {
namespace JSON {

void readConfiguration(const std::string &filename, Config *rootConfig);
void readBufferConfiguration(const std::string &source, Config *rootConfig);
void readNode(const std::string &key, const rapidjson::Value &value, Config *config);

void writeConfiguration(const std::string &filename, const Config *rootConfig, bool prettify = true);
void writeBufferConfiguration(std::string &source, const Config *rootConfig);
void writeNode(const Config *config, rapidjson::Value &rootJSONData, rapidjson::Document::AllocatorType &allocator);

std::string decodeValue(const rapidjson::Value &value);
Expand Down
105 changes: 105 additions & 0 deletions src/IO/configuration_XML.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
#include <libxml/parser.h>
#include <stdexcept>
#include <string>
#include <cstring>

#include "configuration_XML.hpp"

Expand Down Expand Up @@ -216,6 +217,41 @@ void readConfiguration(const std::string &filename, const std::string &rootname,
xmlCleanupParser();
}

/*!
Read xml information from a plain xml-formatted std::string and fill the Config tree.
The method is meant for general-purpose xml info absorbing. So no version checking
is done in this context, nor rootname one.
\param source string containing all the info formatted in XML style.
\param rootConfig pointer to Config tree to store data parsed from the string.
*/
void readBufferConfiguration(const std::string &source, Config *rootConfig)
{
if (!rootConfig) {
throw std::runtime_error("XML::readConfiguration Null Config tree structure passed");
}

// Macro to check API for match with the DLL we are using
LIBXML_TEST_VERSION

// Read the XML string
const char * cstr = source.c_str();
xmlDoc *doc = xmlParseMemory(cstr, strlen(cstr));
if (doc == nullptr) {
throw std::runtime_error("Could not parse XML configuration string: \"" + source + "\"");
}

// Get the root element
xmlNode * rootElement = xmlDocGetRootElement(doc);

//read it as usual
readNode(rootElement->children, rootConfig);

// Clean-up
xmlFreeDoc(doc);
xmlCleanupParser();
}


/*!
Write the configuration to the specified file.

Expand Down Expand Up @@ -283,6 +319,75 @@ void writeConfiguration(const std::string &filename, const std::string &rootname
xmlFreeTextWriter(writer);
}

/*!
Write the Config Tree to a c++ string (xml-stringfication). All contents will
be appended to the target source string.
The method is meant for general-purpose xml info flushing.
\param source string to write to
\param rootConfig pointer to the Config tree to be stringfied.
\param rootname (optional) name of the root section. Default is "root".
*/
void writeBufferConfiguration(std::string &source, const Config *rootConfig, const std::string &rootname)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to add a private function that takes in input a xmlTextWriterPtr? This will allow to share code between writeConfiguration and writeBufferConfiguration? For "symmetry", this apply also to "readBufferConfiguration".

In bitpit , arguments that are modified by the function are usually passed by pointer at the end of the argument list.

Do we really need a function with a different name? The signature of writeConfiguration and writeBufferConfiguration is different, we can use the same name.

I would still pass a "format" argument (also in the read* function). In this way the function that write the configuration to file and the one that write the configuration to string will generate the same output.

Copy link
Contributor Author

@roccoarpa roccoarpa May 31, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right a compact code is always better. But since pending pull #267 and needs of issue #269 suggest a major underhood rework of tree interface and its parsing capabilities, and the introduction of rapidXML will make a lot easier to compact the code w.r.t. to libxml2, I think it's not the case to lose much time on this now.

{
if (!rootConfig) {
throw std::runtime_error("XML::writeConfiguration Null Config tree structure passed");
}

int status;

xmlBufferPtr buffer = xmlBufferCreate();
if (buffer == NULL) {
throw std::runtime_error("Error creating the writing buffer");
}
// Create a new XmlWriter for DOM tree acting on memory buffer, with no compression
xmlTextWriterPtr writer = xmlNewTextWriterMemory(buffer, 0);
if (writer == NULL) {
throw std::runtime_error("Error creating the xml buffer writer");
}
//no indent.
xmlTextWriterSetIndent(writer, 0);

// Start the document
status = xmlTextWriterStartDocument(writer, NULL, DEFAULT_ENCODING.c_str(), NULL);
if (status < 0) {
throw std::runtime_error("Error at xmlTextWriterStartDocument");
}

// Start the root element
xmlChar *elementName = encodeString(rootname, DEFAULT_ENCODING);
status = xmlTextWriterStartElement(writer, BAD_CAST elementName);
if (status < 0) {
throw std::runtime_error("Error at xmlTextWriterStartElement");
}

// Attribute version is not relevant in this context.

// Write the configuration
writeNode(writer, rootConfig, DEFAULT_ENCODING);

// End section
status = xmlTextWriterEndElement(writer);
if (status < 0) {
throw std::runtime_error("Error at xmlTextWriterEndElement");
}

// Close the document
status = xmlTextWriterEndDocument(writer);
if (status < 0) {
throw std::runtime_error("Error at xmlTextWriterEndDocument");
}

// free the XML writer
xmlFreeTextWriter(writer);

//buffer is still hanging on, append its contents to the output string
//xmlChar (aka unsigned char) simple casting to char should be enough
source += std::string(reinterpret_cast<char*>(buffer->content));

//free the buffer
xmlBufferFree(buffer);
}

}

}
Expand Down
2 changes: 2 additions & 0 deletions src/IO/configuration_XML.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -38,9 +38,11 @@ namespace XML {
extern const std::string DEFAULT_ENCODING;

void readConfiguration(const std::string &filename, const std::string &rootname, bool checkVersion, int version, Config *rootConfig);
void readBufferConfiguration(const std::string &source, Config *rootConfig);
void readNode(xmlNodePtr root, Config *config);

void writeConfiguration(const std::string &filename, const std::string & rootname, int version, const Config *rootConfig);
void writeBufferConfiguration(std::string &source, const Config *rootConfig, const std::string &rootname = "root");
void writeNode(xmlTextWriterPtr writer, const Config *config, const std::string &encoding = DEFAULT_ENCODING);

xmlChar * encodeString(const std::string &in, const std::string &encoding);
Expand Down
Loading