Convert from RapidJson to Nlohmann

[paulocoutinhox]

Hi,

How can i convert my code from rapidjson to nlohmann?

rapidjson::Document json;
json.Parse(httpResponse.body.c_str());

if (json.HasMember(fieldName.c_str()))
{
    const rapidjson::Value &value = json[fieldName.c_str()];

    if (value.IsArray())
    {
        auto accessPackages = std::vector<AccessPackage>{};

        for (auto &valueAccessPackage : value.GetArray())
        {
            if (valueAccessPackage.IsObject())
            {
                accessPackages.push_back(fromJsonObject(valueAccessPackage));
            }
        }

        return accessPackages;
    }
}

return {};

My current stage is this (i don't know if is the better way):

auto json = nlohmann::json::parse(httpResponse.body);

if (json.contains(fieldName))
{
    auto value = json[fieldName];

    if (value.is_array())
    {
        auto accessPackages = std::vector<AccessPackage>{};

        for (auto &valueAccessPackage : value.get<???>())
        {
            if (valueAccessPackage.is_object())
            {
                accessPackages.push_back(fromJsonObject(valueAccessPackage));
            }
        }

        return accessPackages;
    }
}

return {};

And how i can iterate when detect an array like in this line for (auto &valueAccessPackage : value.get<???>())?

Thanks.

[falbrechtskirchinger]

Have a look at https://json.nlohmann.me/features/arbitrary_types/.

With conversion functions for AccessPackage, you can write the following:

auto accessPackages = std::vector<AccessPackage>{};
accessPackages.reserve(value.size());

for (auto &valueAccessPackage : value)
{
    if (valueAccessPackage.is_object())
        accessPackages.emplace_back(valueAccessPackage.get<AccessPackage>());
}

[gregmarr]

With conversion functions for AccessPackage, you can write the following:

With the conversion functions, you should be able to just do this, unless you really need the is_object() check for every element in the array because your array elements may not always be objects and you want to silently skip them if they're not.

if (value.is_array())
{
    return value.get<std::vector<AccessPackage>>();
}

[paulocoutinhox]

Nice.

But conversion can be more "smart"? Because in my case, the API sometimes return the same field as string and as int64_t, for example.

Can i need check first if it is string or int64_t before get the value.

Is possible do it on from_json method?

[gregmarr]

Yes, if you write from_json yourself, as opposed to using the automated creation using macros, then you can check the types in the json as you are extracting.

[gregmarr]

Yes, converting that to from_json form should be fine.

[paulocoutinhox]

This is the correct way?

#include "nlohmann/json.hpp"
#include "ubook/domain/AccessPackage.hpp"

namespace ubook
{
namespace domain
{

void to_json(nlohmann::json &j, const AccessPackage &o)
{
    j = nlohmann::json{
        {"id", o.id},
        {"name", p.name},
    };
}

void from_json(const nlohmann::json &j, AccessPackage &o)
{
    // id
    if (j.contains("id"))
    {
        auto &v = j["id"];

        if (v.is_string())
        {
            o.id = std::stoll(v.get<std::string>());
        }
        else if (v.is_number_integer())
        {
            o.id = v.get<int64_t>();
        }
    }

    // name
    if (j.contains("name"))
    {
        auto &v = j["name"];

        if (v.is_string())
        {
            o.name = v.get<std::string>();
        }
    }
}

} // namespace domain
} // namespace ubook

[falbrechtskirchinger]

Yes. (Assuming this is your namespace.)

[paulocoutinhox]

Thanks.

[paulocoutinhox]

Im trying use it as array/vector as suggested here. But i get error:

std::vector<AccessPackage> AccessPackageHelper::fromHttpResponseWithFieldName(const HttpResponse httpResponse, const std::string &fieldName)
{
    auto json = nlohmann::json::parse(httpResponse.body);

    if (json.contains(fieldName))
    {
        auto values = json[fieldName];

        if (values.is_array())
        {
            return values.get<std::vector<AccessPackage>>();
        }
    }

    return {};
}

#include "nlohmann/json.hpp"
#include "ubook/domain/AccessPackage.hpp"

namespace ubook
{
namespace domain
{

void to_json(nlohmann::json &j, const AccessPackage &o)
{
    j = nlohmann::json{
        {"id", o.id},
        {"name", o.name},
    };
}

void from_json(const nlohmann::json &j, AccessPackage &o)
{
    // id
    if (j.contains("id"))
    {
        auto &v = j["id"];

        if (v.is_string())
        {
            o.id = std::stoll(v.get<std::string>());
        }
        else if (v.is_number_integer())
        {
            o.id = v.get<int64_t>();
        }
    }

    // name
    if (j.contains("name"))
    {
        auto &v = j["name"];

        if (v.is_string())
        {
            o.name = v.get<std::string>();
        }
    }
}

} // namespace domain
} // namespace ubook

Error:

[paulocoutinhox]

It can happen because the function std::vector<AccessPackage> AccessPackageHelper::fromHttpResponseWithFieldName(const HttpResponse httpResponse, const std::string &fieldName) is inside namespace ubook::helper and the from/to json is inside ubook::domain? So, the get function only work inside same namespace?

[paulocoutinhox]

[paulocoutinhox]

[paulocoutinhox]

I found the problem. My structs don't have a default constructor, because it was generate by a tool (djinni).

[falbrechtskirchinger]

Works for me.

https://godbolt.org/z/rh7qqzG4x

(Also, the std::move() is of course not needed. I've removed it above.)

[falbrechtskirchinger]

This is an alternative approach to the free functions. Be sure to place the adl_serializer specialization into the nlohmann namespace.

[paulocoutinhox]

I dont understand one thing. I need replace the current from_json and to_json to use the adl_serializer?

[falbrechtskirchinger]

Use either one, not both. I.e., remove the free functions from_json/to_json and implement the adl_serializer specialization instead.

[paulocoutinhox]

Nice.

It works with the second:

#include "nlohmann/json.hpp"
#include "ubook/domain/AccessPackage.hpp"

using namespace ubook::domain;

namespace nlohmann
{
template <>
struct adl_serializer<AccessPackage>
{
    static AccessPackage from_json(const json &j)
    {
        AccessPackage o{0, ""};

        // id
        if (j.contains("id"))
        {
            auto &v = j["id"];

            if (v.is_string())
            {
                o.id = std::stoll(v.get<std::string>());
            }
            else if (v.is_number_integer())
            {
                o.id = v.get<int64_t>();
            }
        }

        // name
        if (j.contains("name"))
        {
            auto &v = j["name"];

            if (v.is_string())
            {
                o.name = v.get<std::string>();
            }
        }

        return o;
    }

    static void to_json(json &j, AccessPackage o)
    {
        j = nlohmann::json{
            {"id", o.id},
            {"name", o.name},
        };
    }
};
} // namespace nlohmann

[paulocoutinhox]

Thanks, i will migrate to other 20 classes :(

If i have more problems, i post here.

[paulocoutinhox]

One more question about my migration. In rapidjson i need test item by item to check if each item exists and is valid. This was made using this methodology to my application don't crash from invalid json response.

I made a new convertion here but i don't have experience to understand what is more secure in my case:

Version 1:

AdsSystemServiceGetInfoData AdsHelper::fromHttpResponse(const HttpResponse httpResponse, const Response response)
{
    auto json = nlohmann::json::parse(httpResponse.body);

    auto data = AdsSystemServiceGetInfoData{response, false, ""};

    if (json.contains("data"))
    {
        auto & dataValue = json["data"];

        if (dataValue.is_object())
        {
            {
                // show
                if (dataValue.contains("show"))
                {
                    auto & v = dataValue["show"];

                    if (v.is_boolean())
                    {
                        data.show = v.get<bool>();
                    }
                }
            }
            {
                // url
                if (dataValue.contains("url"))
                {
                    auto & v = dataValue["url"];

                    if (v.is_string())
                    {
                        data.url = v.get<std::string>();
                    }
                }
            }
        }
    }

    return data;
}

Version 2:

AdsSystemServiceGetInfoData AdsHelper::fromHttpResponse(const HttpResponse httpResponse, const Response response)
{
    auto json = nlohmann::json::parse(httpResponse.body);

    auto data = AdsSystemServiceGetInfoData{
        response,
        json["data"]["show"].get<bool>(),
        json["data"]["url"].get<std::string>(),
    };

    return data;
}

[paulocoutinhox]

One example of problem.

If data.show is not boolean, but a string in an invalid json example. What i more secure to do?
Or data.url is null or a number or not exists. What i more secure to do?

Thanks.

[falbrechtskirchinger]

You could use exception handling with version 2. You may also want to replace operator[] with at() (operator[] will insert a non-existent key).

AdsSystemServiceGetInfoData AdsHelper::fromHttpResponse(const HttpResponse httpResponse, const Response response)
{
    auto json = nlohmann::json::parse(httpResponse.body);

    try {
        auto data = AdsSystemServiceGetInfoData{
            response,
            json.at("data").at("show").get<bool>(),
            json.at("data").at("url").get<std::string>(),
        };
        return data;
    }
    catch(const json::type_error &) {
        return {response, false, ""};
    }
}

[paulocoutinhox]

But in my two versions, they can generate exception? Or it will assume default values for each type (false for bool and empty for string)?

[falbrechtskirchinger]

Version 1 should not generate exceptions (AFAICT), version 2 can.

[paulocoutinhox]

More one thing.

I create from/to json for almost models here. I need use try in this case?

if (j.contains("libraries"))
{
    try
    {
        o.libraries = j["libraries"].get<std::vector<ProductLibrary>>();
    }
    catch (std::exception e)
    {
        // ignore
    }
}

I can use only this?

if (j.contains("libraries"))
{
    o.libraries = j["libraries"].get<std::vector<ProductLibrary>>();
}

[falbrechtskirchinger]

In your example, operator[] will not throw and neither should get() (whether get() throws depends on the implementation of from_json()). In short, no try/catch is necessary.

[paulocoutinhox]

Thanks.

[paulocoutinhox]

I want to thank the @falbrechtskirchinger and @gregmarr for the help.

Our library was fully migrated to nlohmann with success.

We don't have performance problems on mobile and everything is working nice.

The development time with nlohmann is lower and better compared to other libraries.

The possibility to create a custom to_json and from_json help me a lot and make easier convert all APIs jsons from text to our models.

Thank you guys.

[nlohmann]

What do you mean with bigger development time?

[paulocoutinhox]

Sorry, my mistake. I mean lower and better.

[nlohmann]

Cool, thanks for sharing!

[gregmarr]

You're welcome, glad it all worked out for you!