Neural Rendering (AI Project)

[brlcad]

Outline

Use neural net during ray tracing as an acceleration method.

Details

Ray tracing involves computationally expensive calculations and geometry with varying degrees of complexity. The idea for this project is to use neural networks during 3D rendering as an acceleration method.

This work is intended to build on research AMD presented in 2023 High Performance Graphics (Fujieda et. al) and later worked on in BRL-CAD by Texas A&M students. The general idea is to train a neural net on a model that is otherwise expensive to render (e.g., a NURBS model or complex hierarchical model), then use that neural net for ray intersection queries. The hypothesis being explored is whether neural nets can be leveraged for arbitrary shot line queries, accurately and more efficiently than via direct computation.

Expected Outcome

The expected outcome is subject to discussion but probably involves a suite of performance testing, and integration of a trained neural net with BRL-CAD ray tracing.

Project Properties

AI/ML, neural networks, ray intersection testing, high performance graphics

Skills

C/C++

Difficulty

Hard

Size

long (350h)

Additional Information

• Potential mentor(s): Sean Morrison
• Organization website: https://brlcad.org
• Communication channels: https://brlcad.zulipchat.com
• https://brlcad.org/design/neural/

[matthewjselby]

Hi there! This sounds like a really interesting problem to work on. Do you have any further information about the work already done by the Texas A&M students? I'm curious whether there is existing code to build on or if this is proposing a solution from start to finish. Thanks!

[brlcad]

HI @matthewjselby and thanks for your interest! This is definitely an interesting and current area of research. I have a lot of info from our TAMU students, but still need to upload it somewhere reachable. There is both opportunity to build on existing code, use their code as reference, or propose a solution from start to finish. It really will depend on your experience and the goals set forth. Can you share some of your background working with NNs?

[matthewjselby]

@brlcad I've touched on some of the theory behind NNs in my classes (I'm currently a masters student at Rice in CS), but haven't yet had the opportunity to get much hands-on experience working with them, despite wanting to. I see this as an opportunity to do just that, but I would be interested in your thoughts as to whether my level of experience is compatible with the project. I feel relatively comfortable in C/C++ and my ability to get up to speed quickly on new technologies. Let me know what you think!

[brlcad]

@matthewjselby It's certainly a possibility but will of course just depend on how much scope you propose and how quickly you can get up-to-speed on training. Here is the research paper that this work is partially based on: https://gpuopen.com/download/publications/HPG2023_NeuralIntersectionFunction.pdf

If you can understand a good bit of how they did what they did and, better still, are able to reproduce their work, then you'd probably be just fine proposing this task. If not, no worries -- I'm sure someone with your abilities will be able to scope a project adequately. Look forward to hearing more of your thoughts and background. You're encouraged to join our Zulip chat for additional discussion opportunities.

[iOmkarNikam]

Hello Mentors Team BRL-CAD !
@brlcad [Sean Morrison]

I'm Omkar Nikam ,I am second year student studying Artificial Intelligence and Machine Learning (CSE) from KITCOEK(INDIA).
I am Very Keen Learner , My Expertise Include Python , C , C++ , JAVA , HTML, CSS , JavaScript , PHP .

I'm beginner in Open Source , I'm Passionate for Technology .
I have read and gone through the description , And I am interested to work with BRL-CAD Projects for GSOC 2024 ,
particularly in ;
Neural Rendering (AI Project) (#90)

I am having good exposure to Machine Learning Concepts .
Please , guide me for next steps !

Sincerely ,
Omkar

[brlcad]

Hi @iOmkarNikam and thanks for the intro here and via e-mail. Please check out the AMD HPG paper and join us on Zulip chat to discuss.

[ZhejiaHu]

Hello. I am computer science student from Delft University of Technology and I am quite interested in the neural tracing project affiliated of GSOC 2024? I have project experiences with both neural network, which is a competition, and projects with top design companies in China. I have got full marks in my Computer Graphics course. If you need more information on me, feel free to private message me and I will send you the CV and I am also happy to discuss with you the contents of the paper.

[brlcad]

Additional information relevant to this project idea is available at https://brlcad.org/design/neural/