Port Existing MRPT algorithms for V-REP simulator (shubham-kumar1410)

[bergercookie]

Initial Description

V-REP provides a good platform for combining external libraries that are often used in robotic simulations. The aim of the project is to use this flexibility of the simulator and port existing MRPT algorithms for V-REP simulator so that they can be used in the V-REP simulator. The libraries will be extended to V-REP with the help of V-REP Remote Api which is part of the V-REP API framework

Original GSoC Proposal

Progress: All comments below reflect the interactions between the GSoC student and the mentors.

---

• Student: @shubham-kumar1410
• Main Mentor: @bergercookie
• Backup Mentors: @maxchaos, @EduFdez

[bergercookie]

Hey @shubham-kumar1410, @ktushar14,

Welcome to MRPT and to GSoC!

Along with @maxchaos, @EduFdez, @taihup we will be mentoring your projects and try to provide guidance and advice whenever possible.

In sake of starting this discussion and since this is the bonding period, could you please give a small introduction about you so that the MRPT mentors and also @rachit173, @karnikram get to know you better?

Regards,
Nikos

P.S. I'm writing to this issue so that I don't repeat myself but feel free to comment in your corresponding tickets.

[shubham-kumar1410]

Hi. I am Shubham Kumar and I am currently a 3rd year student of BITS Pilani KK Birla Goa Campus,India majoring in Electrical and Electronics Engineering.
Thanks for providing me the opportunity to work on this project and I hope we have a great summer together.
I have started looking at the MRPT algorithms in order to familiarise myself with the codebase as well as the algorithm. I have even finished building small remote apis for V-REP ( https://github.com/shubham-kumar1410/VREP_Remote_Api_Test ).
I am considering using the plugin already built (port : 19997) for the server side. Please let me know if this is fine or should I create a new plugin for the same.
I am reading about building external projects using ExternalProject_Add currently. Please let me know if there is anything else I need to read about.

Regards,
Shubham.

[maxchaos]

Hi to all students and congratulations for your acceptance!
Along with @bergercookie and @EduFdez, I will be working in the V-Rep simulator project as a backup mentor. As I have some experience with V-Rep, I will try as best as I can to help out with any questions related to that which may arise.
@shubham-kumar1410, I really like your enthusiasm and I hope we will all have a creative and productive summer!

[bergercookie]

Hey @shubham-kumar1410,

I have started looking at the MRPT algorithms in order to familiarise myself with the codebase as well as the algorithm.

Great, don't hesitate to ask any questions you may have with the MRPT APIs, CMake configuration or anything else you may think of.

I am considering using the plugin already built (port : 19997) for the server side. Please let me know if this is fine or should I create a new plugin for the same.

Hmmm, which plugin are you referring to?

I am reading about building external projects using ExternalProject_Add currently. Please let me know if there is anything else I need to read about.

Yeah, knowing how to handle this is going to be important in the project.

All in all try to familiarise with the way MRPT is organised and works, e.g., cmake structure, coding standards, .ini config files etc. Take a look and make sure that you 're comfortable with the various tools we'll be using: Git for version control, Doxygen for documentation, GTest for unittests. Also have a read through the V-REP documentation and examples for the Plugins and the External API.

Take your time with that and probably next week we can setup a call along to discuss and set specific goals and milestones for the project. How does that sound?

Regards,
Nikos

[shubham-kumar1410]

Hi @bergercookie @maxchaos thanks for your response and I am really looking forward to working with you guys.
A default remote api server service runs on the port 19997 and it is loaded when the simulator is launched. (http://www.forum.coppeliarobotics.com/viewtopic.php?t=5197 ). It worked well for my sample remote api.
I will try to familiarise myself with the code style of MRPT as soon as possible. Can we schedule our call on next weekend as during the week I have my end semester exams which ends on next Friday ?

Regards,
Shubham

[ktushar14]

Hey everyone, please take a look at my post in my corresponding issue.

[bergercookie]

I will try to familiarise myself with the code style of MRPT as soon as possible. Can we schedule our call on next weekend as during the week I have my end semester exams which ends on next Friday ?

Sure, let's pick it up then.

[EduFdez]

Hi @bergercookie !
The link of the GSoC proposal in the 1st post of this thread is only available to mentors. Can you move it to a public place (e.g. Google drive) and update the link? Please.
PS. same for the GTSAM project.

[shubham-kumar1410]

Hi,
Due to my university exams in the previous week I was unable to do a lot of work. Still I managed to test out some more V-REP remote api functions and I am fairly comfortable with them right now.
Currently I am going through the graphslam ROS package to familiarise myself with the algorithm.
I have one more exam on Monday (6/5/2018) and after that I will try finishing these tasks as soon as possible.
@bergercookie I went through the mrpt_vrep_bridge repository. If possible kindly add some issues so that I can work with them.
Also kindly let me know when would you all like to schedule our call .

[bergercookie]

Hey @shubham-kumar1410,

@bergercookie I went through the mrpt_vrep_bridge repository. If possible kindly add some issues so that I can work with them.

Yes, so just to get everyone in the conversation up-to-date:

I created a template repository mrpt_vrep_bridge which should act as the basis of this GSoC project. It contains boilerplate code (mostly cmake configuration) for using Doxygen for documentation and GTest for unittesting, as well as a basic file structure of how I'd like the final work to be.

For now start by adding cmake support for the MRPT library as well as VREP.
The former should be quite straightforward with find_package. See the mrpt-ros-pkg algorithms as an example.

The latter may need some thought. If I were you I would use an environment variable to specify where V-REP is located in the filesystem (e.g., VREP_PATH). Then I would use that to let cmake know where to find the header files (include_directory) and what libraries to link against (target_link_libraries).

Try building a plugin of yours using this setup and a sample remoteApi application.

After compiling the vrep-dependent targets you'll have to cmake-install the latter (at least the generated .so files) to the vrep root directory itself as plugins cannot be loaded otherwise.

Also kindly let me know when would you all like to schedule our call .

How about Tuesday 3:00 BST?

[jolting]

@bergercookie Put it here: https://github.com/MRPT/mrpt_vrep_bridge

[bergercookie]

UPDATE: Migrated mrpt_vrep_bridge - https://github.com/MRPT/mrpt_vrep_bridge

[shubham-kumar1410]

I'll start by adding cmake support.
Tuesday 3:00 pm seems fine.

[shubham-kumar1410]

As decided in my call with @bergercookie and @maxchaos I will read up about cmake and try building the remote test api without qmake. Once that is done, I will try to retrieve data from a V-REP scene and convert it to mrpt class objects so that it can easily be processed by the library. This part will be common for porting all libraries and hence it is given a lot of preference

[bergercookie]

Hey guys,

I added the general milestones we want to reach as well as the intermidiate / smaller goals that comprise them. @shubham-kumar1410, @maxchaos take a look at the timeline and tell me whether it makes sense for you guys. The point is to have a basic functional prototype at the end of phase 1 of mrpt-graphslam running in V-REP and build on top of it / polish / extend it in the second.

• Milestones
• Issues list

P.S. @jlblancoc Zenhub throws me errors when I want to assign issues to somebody or when I want to add them to a milestone etc. Is there any extra configuration step that I should take before using it or is it a bug on their side?

[shubham-kumar1410]

I found a way to build to build the remote api using cmake by adding
execute_process (COMMAND "/bin/bash" "-c" "cd ${VREP_ROOT}/Vrep/programming/v_repExtApi; qmake v_repExtApi.pro; make") to the cmake file.
I am working on this issue right now and I will try to complete this as soon as possible

[bergercookie]

Hey Shubham,

I found a way to build to build the remote api using cmake by adding
execute_process (COMMAND "/bin/bash" "-c" "cd ${VREP_ROOT}/Vrep/programming/v_repExtApi; qmake v_repExtApi.pro; make") to the cmake file.

Eeem, no that's not the way to do it... and for a number of reasons. :-)

• You are essentially using CMake as a wrapper around qmake. But qmake and cmake are the same kind of tools i.e., they generate UNIX Makefiles that you use to run make.
• This is not a portable solution, that is it runs only in UNIX systems where /bin/bash is available

---

Let me break it down..

What's your goal? To build the remoteApi client application of yours. To do that you need to let it know of the locations of some very specific .h, .c files of V-REP, that's all.

First of all run qmake in your client application directory to generate a template Makefile and then run make on that (from the command line). When you run make you can see exactly what directories are used to search for header files. These directories should be added in the include_directories call of CMake. Now for the implementation of files of the extApi (extApi.c, 'extApiPlatform.c, shared_memory.c) you have 2 options..

Either include them directly in your .c file after the .h files which is what the remoteApi client tutorial also recomends or you can most probably compile them once into a shared library .so file and use the latter in the target_link_libraries directive of CMake.

Let me know if all this is clear...

[shubham-kumar1410]

Thanks. I'll try to use this method to build the api.

[shubham-kumar1410]

Update :
I have finished on creating a cmake file which builds a VREP remote api into an executable. I have started working on converting laser scan data received from VREP into CObservation2DRangeScan class objects which can be processed by the mrpt libraries.

[shubham-kumar1410]

Update:
I have finished the conversion of vrep laser scans to CObservation2DRangeScan class objects. I have pushed the same on a new branch(link for the method). I will send a pr once all the conversions are completed.

[bergercookie]

Hey @shubham-kumar1410,

I've taken a look at your new commits. To make the reviewing in a more organised manner make a PR from your vrep_convert branch to the newly pushed upstream mrpt_vrep_bridge branch

[bergercookie]

UPDATE:

• Pull-Request: Add VREP conversion methods for MRPT class objects mrpt_vrep_bridge#13
  It contains a series of issues that need to be addressed in the existing code

[shubham-kumar1410]

I have looked at your PR review and I am working on them now. I will send the commit once I have addressed all the review points.

[shubham-kumar1410]

Update :
I have made almost all the changes as requested as well as the basic documentation for the conversion methods. I will keep updating the documentation as we progress.

[shubham-kumar1410]

Update :
I have started working on the 2d graphslam remote api. In the next few days I will be going through the library and also will be working on the api along the way.

[shubham-kumar1410]

Hi. The _execGraphSlamStep method uses 4 arguments : CActionCollection::Ptr action,CSensoryFrame::Ptr observations,CObservation::Ptr observation and a size_t variable. I am not sure how they work. I understood that CSensoryFrame object contains the CObservation2DRangeScan values and the CActionCollection object the odometry values. Can you help me with the roles of the other two variables ? I tried going through the code but I am not clear on this.

[bergercookie]

Hey @shubham-kumar1410,

I added some additional comments on the pending PR of yours.

The _execGraphSlamStep method uses 4 arguments : CActionCollection::Ptr action,CSensoryFrame::Ptr observations,CObservation::Ptr observation and a size_t variable. I am not sure how they work. I understood that CSensoryFrame object contains the CObservation2DRangeScan values and the CActionCollection object the odometry values. Can you help me with the roles of the other two variables ? I tried going through the code but I am not clear on this.

Check this document: https://www.mrpt.org/Rawlog_Format

In each iteration either the CObservation part is filled or both CACtionCollection and CSensoryFrame.

That solely depends on what kind of format you want to use and support. A list of CObservation is much more flexible but in the context of filtering which we don't do in graphslam using action + corresponding observations may be more appropriate.

The last argument is the rawlog entry. It's not that important. Check the code, it's just beign used to update the ground-truth visualisation.

[bergercookie]

@shubham-kumar1410, 1st evaluation starts on the 11th of June. Please submit a PR with a functional example of using graphslam with Input from V-REP whenever you are ready so that we can review it..

[shubham-kumar1410]

I have sent the commit with all the changes requested,kindly review it.
I am unable to pass an empty file name as the rawlog file name in the constructor of CGraphSlamEngine. It is giving me a file not found error in this line. This function is being called in a try catch block but still it is giving an error. Is there a workaround for this ?
I will send a PR by tonight or tomorrow morning there a few things to fix.
Update: The exception has been handled. There was another exception which was causing a segmentation fault. It has been resolved. Sorry for the confusion

[shubham-kumar1410]

I have pushed the graphslam api on this branch. Kindly have a look. I will be pushing some more changes to it soon.

[maxchaos]

Firstly, I believe it is a very bad idea to throw graphslam's headers inside your project like that; the main reason being that these files technically belong to another project (they are maintained elsewhere and should have been installed by whatever package installed mrpt to your os). Normally, you should set up CMake to automatically locate these headers and expose them to your project (in this case, @bergercookie has already done that for you).
Secondly, you should not directly modify graphslam's headers yourself. If, as you mention in your latest commit's message, you did find a bug in graphslam's code, then you should open an issue about it on github and let us know. ;)

[shubham-kumar1410]

Hi. I am sorry for pushing the edited header files to github. I wanted to make sure my api was functioning properly ahead of the deadline hence I panicked and edited the header to check the functionality. I was planning on removing all the headers before I submit the PR. Also , the reason I put the graphslam headers in the include directory is that I don't have them installed along with other MRPT libraries in my system. I had installed the libs using this commandsudo apt-get install libmrpt-dev mrpt-apps.

The API is still a work in progress. It does not take the config file name and the registration deciders,optimiser from the user. I am implementing these things right now along with a couple of changes and I will push these changes soon.

[maxchaos]

Don't worry about it. No harm done.
Regarding the headers, that is strange. I think they should have been installed with the aforementioned packages, normally somewhere under /usr/include/mrpt/graphslam on Ubuntu. What os version do you use?

[shubham-kumar1410]

It is not located there with the rest of the mrpt libs.There are some other libs like gui that are missing. I tried searching on my os but it is not there. I even tried reinstalling the whole package. Hence I added them to the project folder.I am using Ubuntu 16.04 right now.

[bergercookie]

Also , the reason I put the graphslam headers in the include directory is that I don't have them installed along with other MRPT libraries in my system. I had installed the libs using this commandsudo apt-get install libmrpt-dev mrpt-apps.

It is not located there with the rest of the mrpt libs.There are some other libs like gui that are missing. I tried searching on my os but it is not there. I even tried reinstalling the whole package.

Since we're developing on top of MRPT it makes sense to compile the latter from source instead of installing it as a debian package. See the github page for instructions on that. You can't find mrpt-graphslam in MRPT 1.3 which is included by default in 16.04 (https://packages.ubuntu.com/source/xenial/mrpt).

Just for reference though, if you were to use ppas, use these instead:

sudo add-apt-repository ppa:joseluisblancoc/
sudo apt-get update
sudo apt-get install libmrpt-dev mrpt-apps

[bergercookie]

I 'll test the new functionality today and tomorrow, @shubham-kumar1410 please edit the README file for instructions on your current setup (how to run, what to expect, what's currently working and what doesn't)

[shubham-kumar1410]

I will edit the README file with the instructions and also I will look into compiling MRPT directly from the source

[shubham-kumar1410]

I have edited the README file. I am still not clear how will we link the header files from the source to our project. Should I just include the header files located in my MRPT folder cloned from the source or is there any other way ?

[rachittibrewal]

target_link_libraries(project PUBLIC ${MRPT_LIBS})
And yes also,
find_package(MRPT REQUIRED core poses rtti slam....)
Hope this helps.
P.S. Maybe your system is unable to find the MRPT source in that case check the tutorial on compiling first program using MRPT.

[shubham-kumar1410]

I have added all this to the main cmakelists file yet I am unable to find the graphslam headers . I'll try going through the tutorial once. Thanks :)

[bergercookie]

I had a brief look at your code and application... here's some thoughts:

1. We want to have a scene with static objects and a moving sensor, ideally a robot with a sensor / set of sensors mounted on it. Having a static laser scanner on the floor simply won't cut it for a SLAM algorithm

2. With regards to this snippet

simxGetObjectHandle(clientID,"fastHokuyo",&handle,simx_opmode_streaming);
simxGetObjectPosition(clientID,handle,-1,position,simx_opmode_streaming); // get object orientation(returns euler angles)
simxGetObjectQuaternion(clientID,handle,-1,quaternion,simx_opmode_streaming);
...
CPose3D sensor_pose;
bool pose_convert = convert(position,quaternion,sensor_pose);
CObservationOdometry odom;
bool odometry_convert = convert(sensor_pose, vel, angularvel,odom);
CPose2D sensor_pose_2d = odom.odometry;

This is wrong. graph-slam operates with odometry measurements (e.g., from the mobile robot wheels) + laser scan measurements

Using the V-REP provided correct position and orientation of the laser scanner object as odometry measurements wont do. There should be an actual robot there with wheels where we'd be able to use an odometry or velocity model. Then on that measurement we should apply a gaussian / banana / triangular noise distribution and use that as input to graphslam.

3. Up to this point I expected something more generic that could potentially be extended to support other MRPT algorithms. Instead this is rather fragile script and at this point too tightly coupled to the graphslam algorithm which would potentially take considerable amount of time to refactor into a generic API.

All in all, I consider this a very good effort on your part but still it's not enough to go on to the 2nd phase of GSoC and come up with the expected results by the end of that.

Let me know if you have any questions regarding all this either in this issue or in the chat...

[shubham-kumar1410]

Hi @bergercookie , thanks for your feedback. I understand that a static laser won't cut . Earlier I had started working on creating a scene with a Rover with a scanner mounted on top of it. I thought it would be better if I start working on the scene once I have ported one library and that's what I am doing right now.
Hence I am calculating the odometry of the laser itself. But as rightly pointed out by you we should do it on the wheels of the robot.
I think the api can be ported to any library which uses Cobservation2DRangeScan and COdometryObservation class objects. Since those objects are created using the V-REP measurements , I think we can port similar libraries since the V-REP measurements part is already handled. Correct me if I am missing something.

[shubham-kumar1410]

Hi, there are a couple of things which I didn't understand. The current odometry measurements considers the robot position and quaternion to create the COdometryObservation class object. Is there something that I am missing ? Also I didn't quite understand the contents of the ground truth file. I mean that what are the contents of it

I would like to work on this project irrespective of the evaluation result and would be grateful if this issue is open for my queries.