Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up gazebo error/warning messages #109

Open
emailweixu opened this issue Nov 26, 2019 · 11 comments
Open

Clean up gazebo error/warning messages #109

emailweixu opened this issue Nov 26, 2019 · 11 comments
Assignees

Comments

@emailweixu
Copy link
Contributor

emailweixu commented Nov 26, 2019

There are a lot of gazebo error/warning messages. They are confusing the new users. We should find ways to avoid them:

[Err] [ConnectionManager.cc:472] Data was empty, try again
[Err] [Connection.hh:268] Header is empty
[Err] [LogRecord.cc:158] LogRecord has not been initialized.
Warning [parser.cc:531] Converting a deprecated SDF source[data-string].
[Wrn] [Publisher.cc:142] Queue limit reached for topic /gazebo/default/pose/local/info, deleting message. This warning is printed only once.
@Jialn Jialn self-assigned this Nov 26, 2019
@Jialn
Copy link
Contributor

Jialn commented Nov 26, 2019

add another one:

[Wrn] [Event.cc:61] Warning: Deleting a connection right after creation. Make sure to save the ConnectionPtr from a Connect call

@Jialn
Copy link
Contributor

Jialn commented Nov 26, 2019

"LogRecord has not been initialized" can be fixed by add initialization to LogRecord. Will fix it soon.

@Jialn
Copy link
Contributor

Jialn commented Nov 26, 2019

"Warning [parser.cc:531] Converting a deprecated SDF source[data-string]"

The warning is generated at "sdf.SetFromString(sdfString)" at line 462 of pygazebo.cc. SetFromString():
https://bitbucket.org/osrf/gazebo/src/eae3d1b7adce37b2838501a25e42d36966dce298/gazebo/sdf/interface/SDF.cc#lines-1641
It is printed at line 361 at "/gazebo/sdf/interface/parser.cc":
https://bitbucket.org/osrf/gazebo/src/eae3d1b7adce37b2838501a25e42d36966dce298/gazebo/sdf/interface/parser.cc#lines-361

Any insetting model or loading world operation using a sdf string lower than 1.6 (most of our models are in version 1.5 or 1.4) will throw this warning, and it seems can not be disabled. We can set the sdf version to 1.6 when inserting to avoid this warning.

@Jialn
Copy link
Contributor

Jialn commented Nov 26, 2019

[Err] [ConnectionManager.cc:472] Data was empty, try again
[Err] [Connection.hh:268] Header is empty

Where does this come out? I haven't see them on my ubuntu PC.

@emailweixu
Copy link
Contributor Author

emailweixu commented Nov 27, 2019

[Err] [ConnectionManager.cc:472] Data was empty, try again
[Err] [Connection.hh:268] Header is empty

Where does this come out? I haven't see them on my ubuntu PC.

I have a lot of these messages during the training like the following:

I1126 16:40:30.009263 140181041379136 policy_trainer.py:323] 164 time=8.522 throughput=1502.07
[Err] [Connection.hh:268] Header is empty
[Err] [ConnectionManager.cc:472] Data was empty, try again
[Err] [Connection.hh:268] Header is empty
[Err] [ConnectionManager.cc:472] Data was empty, try again
[Err] [Connection.hh:268] Header is empty
[Err] [ConnectionManager.cc:472] Data was empty, try again
I1126 16:40:38.452174 140181041379136 policy_trainer.py:323] 165 time=8.440 throughput=1516.63
I1126 16:40:47.059675 140181041379136 policy_trainer.py:323] 166 time=8.605 throughput=1487.51

@Jialn
Copy link
Contributor

Jialn commented Nov 27, 2019

[Err] [LogRecord.cc:158] LogRecord has not been initialized.
Warning [parser.cc:531] Converting a deprecated SDF source[data-string].

These 2 have been sovled in #110

@Jialn
Copy link
Contributor

Jialn commented Nov 27, 2019

[Err] [ConnectionManager.cc:472] Data was empty, try again
[Err] [Connection.hh:268] Header is empty
[Wrn] [Publisher.cc:142] Queue limit reached for topic /gazebo/default/pose/local/info, deleting message. This warning is printed only once.
[Wrn] [Event.cc:61] Warning: Deleting a connection right after creation. Make sure to save the ConnectionPtr from a Connect call

Now we have these 4 left. All have been related to the transporting module of gazebo. I have try to solve them but failed. Here are some findings during the progress.

[Err] [ConnectionManager.cc:472] Data was empty, try again is caused by "ConnectionManager::OnRead()" in ConnectionManager.cc, for some unknown reason it receive a empty packet. "Header is empty" is similar, "OnReadHeader()" in Connection.hh.

[Wrn] [Event.cc:61] Warning: Deleting a connection right after creation. Make sure to save the ConnectionPtr from a Connect call this one is printed in Event.cc

Connection::~Connection()
{
  common::Time diffTime = common::Time::GetWallTime() - this->creationTime;
  if ((this->event && !this->event->Signaled()) &&
      diffTime < common::Time(0, 10000))
  {
    gzwarn << "Warning: Deleting a connection right after creation. "
          << "Make sure to save the ConnectionPtr from a Connect call\n";
  }

for some unknown reason one connection is deleted right after creation.

[Wrn] [Publisher.cc:142] Queue limit reached for topic /gazebo/default/pose/local/info, deleting message. This warning is printed only once.
The topic "/gazebo/default/pose/local/info" is very frequent compared to other topics, published at thousands Hz, and several MB/s bandwidth. these message are published in "world::RunBlocking()" at each step of world and no option to turn it off.

The call path to setting up transport is:
gazebo::setupServer() -> gazebo_shared::setup() -> gazebo::transport :init() and gazebo::transport :run()

"gazebo::transport :init()" : get master uri, topic manager init, connection manager init
"gazebo::transport:run()" : run transport loop, start a thread transport::connectionmanager::run, update and process master messages

I've tried "gazebo::transport::clearbuffer()" at each step, does not help.
I also tried set "gazebo::transport::stop()" right after "gazebo::setupServer()" to force gazebo stop transporting. More warnings and errors occur and gzclient fails to open. But the simulation is still working and the performance improves a little bit (Agent pioneer, average fps 170 -> 185).

@emailweixu
Copy link
Contributor Author

The other error messages only display at the initial phase the training. But the following two are all over the whole training process:

[Err] [ConnectionManager.cc:472] Data was empty, try again
[Err] [Connection.hh:268] Header is empty

According to the comment in World.cc, /gazebo/default/pose/local/info is needed for rendering. So although simulation can still work, perhaps rendering will fail it is disabled?

@Jialn
Copy link
Contributor

Jialn commented Nov 28, 2019

Which environment or gin file can generate the 2 type of messages? I have't find them. Let me reproduce first.

[Err] [ConnectionManager.cc:472] Data was empty, try again
[Err] [Connection.hh:268] Header is empty

@Jialn
Copy link
Contributor

Jialn commented Dec 4, 2019

found these Errs when running PR2 gripping task with some possibilities.

@andrestoga
Copy link

andrestoga commented Dec 23, 2022

Any idea why the [Wrn] [Event.cc:61] Warning: Deleting a connection right after creation. Make sure to save the ConnectionPtr from a Connect call is happening?

Is it something to be worried about?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants