Model rendering and annotation tools

This is a code pool of Yida Wang, master student of Dr. Weihong Deng. Those codes are used for model rendering from 3D models to 2D synthetic images together with some specific annotation and contour generation. It's basically written in c++, but there are also some basic scripts for API.

Compiling is based on cmake and make, it's designed for Linux, MacOS and Unix.

Author information

Yida Wang

PhD candidate with Dr. Federico Tombari in Technische Universität München

M.Eng with Dr. Weihong Deng in Beijing University of Posts and Telecommunications

Email Address: [email protected], [email protected]

ResearchGate, Github, GSoC 2016, GSoC 2015

Publications

1. ZigzagNet: Efficient Deep Learning for Real Object Recognition Based on 3D Models (ACCV 2016)

• Self-restraint Object Recognition by Model Based CNN Learning (ICIP 2016)

• Face Recognition Using Local PCA Filters (CCBR 2015)

• CNTK on Mac: 2D Object Restoration and Recognition Based on 3D Model (Microsoft Open Source Challenge 2016)

• Large-Scale 3D Shape Retrieval from ShapeNet Core55 (EG 2016 workshop: 3D OR)

Code samples

Details for camera position

Regular objects on the ground using a semisphere view system

if (semisphere == 1)
{
    for (int pose = 0; pose < static_cast<int>(campos_temp.size()); pose++)
    {
      if (campos_temp.at(pose).z >= -0.3 && campos_temp.at(pose).z < z_range)
        campos.push_back(campos_temp.at(pose));
    }
}

Special object such as plane using a full space of view sphere

else
{
  for (int pose = 0; pose < static_cast<int>(campos_temp.size()); pose++)
  {
  if (campos_temp.at(pose).z < 0.3 && campos_temp.at(pose).z > -0.8)
  campos.push_back(campos_temp.at(pose));
  }
}

Model file searching

List the file names under a given path

listDir(bakgrdir_p.c_str(), name_bkg_p, false);
for (unsigned int i = 0; i < name_bkg_p.size(); i++)
{
  name_bkg_p.at(i) = bakgrdir_p + name_bkg_p.at(i);
}

Start rendering

Step 1: Reader is the tool to load the poly files

vtkSmartPointer<vtkOBJReader> reader =
  vtkSmartPointer<vtkOBJReader>::New();
reader->SetFileName(objmodel.c_str());
reader->Update();

Step 2: Build visualization enviroment: Mapper loads what reader recorded

vtkSmartPointer<vtkPolyDataMapper> ObjectMapper =
  vtkSmartPointer<vtkPolyDataMapper>::New();
ObjectMapper->SetInputConnection(reader->GetOutputPort());

Step 3: Actor set target on Mapper, this is what we use for further operation on mesh

vtkSmartPointer<vtkActor> actor =
  vtkSmartPointer<vtkActor>::New();
actor->SetMapper(ObjectMapper);

Step 4: Start the main task for rendering Step 4(1): Render operates on -> Actor

vtkSmartPointer<vtkRenderer> ren =
  vtkSmartPointer<vtkRenderer>::New();
ren->AddActor(actor);
ren->TexturedBackgroundOn();
ren->GetActiveCamera()->SetViewUp(0,1,0);

Step 4(2): RenderWindow operates on -> Render

vtkSmartPointer<vtkRenderWindow> renWin =
  vtkSmartPointer<vtkRenderWindow>::New();
renWin->AddRenderer(ren);
renWin->SetSize(227,227);

Step 4(3): Interactor operates on -> RenderWindow

vtkSmartPointer<vtkRenderWindowInteractor> iren =
  vtkSmartPointer<vtkRenderWindowInteractor>::New();
iren->SetRenderWindow(renWin);
renWin->Render();

Step 5: Define details for rendering

vtkSmartPointer<vtkJPEGReader> imReader =
  vtkSmartPointer<vtkJPEGReader>::New();
vtkSmartPointer<vtkTexture> atext =
  vtkSmartPointer<vtkTexture>::New();
atext->SetInputConnection(imReader->GetOutputPort());
atext->InterpolateOn();
ren->SetBackground(0,0,0);
ren->SetBackgroundTexture(atext);