diff --git a/README.md b/README.md
index 59e3911..affaf7e 100644
--- a/README.md
+++ b/README.md
@@ -52,12 +52,12 @@ Topics:
|(4)
May 13| **Graphics pipeline**
depth buffer method, shading, shadow, anti aliasing | [task04](task04) | [[12]](http://nobuyuki-umetani.com/acg2024s/rasterization_3d.pdf) [[13]](http://nobuyuki-umetani.com/acg2024s/graphics_pipeline.pdf) |
|(6)
May 20| **Ray Casting 1**
spatial data structure | [task05](task05) | [[14]](http://nobuyuki-umetani.com/acg2024s/shading.pdf) [[16]](http://nobuyuki-umetani.com/acg2024s/implicit_modeling.pdf) |
|(7)
May 27| **Ray Casting 2**
Rendering equation, Monte Carlo integration | [task06](task06) | [[15]](http://nobuyuki-umetani.com/acg2024s/rasterization_subpixel.pdf) [[17]](http://nobuyuki-umetani.com/acg2024s/ray_casting.pdf) [[18]](http://nobuyuki-umetani.com/acg2024s/monte_carlo_integration.pdf) [[19]](http://nobuyuki-umetani.com/acg2024s/ray_triangle_collision.pdf) |
-|(8)
June 3| **Character animation**
Linear blend skinning | [task07](task07) | [[20]](http://nobuyuki-umetani.com/acg2024s/character_deformation.pdf) [[21]](http://nobuyuki-umetani.com/acg2024s/jacobian.pdf) |
+|(8)
June 3| **Character animation**
Linear blend skinning | [task07](task07) | [[21]](http://nobuyuki-umetani.com/acg2024s/jacobian.pdf) |
|(9)
June 10| Guest lecture by Dr. Rex West | | |
-|(10)
June 17| **Optimization**
Inverse kinematic | task08 | |
-|(11)
June 24| Laplacian mesh deformation | task09 | |
-|(12)
July 12| **Grid-based Fluid Ⅰ**
Poisson equation | task10 | |
-|(13)
July 8| **Grid-based Fluid Ⅱ**
Stam fluid | | |
+|(10)
June 17| **Character animation2**
Inverse kinematic | [task08](task08) | [[20]](http://nobuyuki-umetani.com/acg2024s/character_deformation.pdf) |
+|(11)
June 24| **Optimization** | | |
+|(12)
July 12| **Laplacian mesh deformation**
| task09 | |
+|(13)
July 8| **Grid-based Fluid**
Poisson equation, Stam fluid | | |
## Grading
diff --git a/task08/README.md b/task08/README.md
new file mode 100644
index 0000000..e2b6fe3
--- /dev/null
+++ b/task08/README.md
@@ -0,0 +1,114 @@
+# Task08: Skeletal Animation
+
+
+
+**Deadline: May 20th (Thu) at 15:00pm**
+
+----
+
+## Before Doing Assignment
+
+### Install Python (if necessary)
+We use Python for this assignment.
+This assigment only supports Python ver. 3.
+
+To check if Python 3.x is installed, launch a command prompt and type `python3 --version` and see the version.
+
+For MacOS and Ubuntu you have Python installed by default.
+For Windows, you may need to install the Python by yourself.
+[This document](https://docs.python.org/3/using/windows.html) show how to install Python 3.x on Windows.
+
+
+### Virtual environment
+
+We want to install dependency ***locally*** for this assignment.
+
+```bash
+cd acg-
+python3 -m venv venv # make a virtual environment named "venv"
+```
+
+Then, start the virtual environment.
+For Mac or Linux, type
+
+```bash
+source venv/bin/activate # start virtual environment
+```
+
+For Windows, type
+
+```bash
+venv\Scripts\activate.bat # start virtual environment
+```
+
+In the command prompt, you will see `(venv)` at the beginning of each line.
+There will be `venv` folder under `acg-`.
+
+### Install dependency
+
+In this assignment we use many external library. We use `pip` to install these.
+
+```bash
+pip3 install numpy
+pip3 install moderngl
+pip3 install moderngl_window
+```
+
+Alternatively, you can install above dependency at once by
+
+```bash
+cd acg-/task08
+pip3 install -r requirements.txt
+```
+
+type `pip3 list` and then confirm you have libraries such as `moderngl`, `numpy`, `pillow`, `pyglet`, `pyrr` etc.
+
+### Make branch
+
+Follow [this document](../doc/submit.md) to submit the assignment, In a nutshell, before doing the assignment,
+- make sure you synchronized the `main ` branch of your local repository to that of remote repository.
+- make sure you created branch `task08` from `main` branch.
+- make sure you are currently in the `task08` branch (use `git branch -a` command).
+
+Now you are ready to go!
+
+---
+
+## Problem 1 (Python execution practice)
+
+Run the code with `python3 main.py`
+
+The program will output `output.png` that update the image below
+
+
+
+## Problem 2 (Skeletal animation)
+
+In this problem, we compute the 3D transformation (rotation and translation) of each bone.
+
+Bones in a skeleton of a character ave a tree structure. Each bone has parent bone, except for the root bone (the bone with index 0).
+For bone `i_bone`, the index of parent bone is `self.bone2parentbone[i_bone]`.
+3D Affine transformation from the parent bone is written in `bone2relativeTransformation`.
+Note that index of parent bone is always smaller than the child bone e.g., `ibone > bone2relativeTransformation[i_bone]`.
+
+Write some code around line #??? to compute the transformations of all the bones in `bone2globalTransformation`.
+
+This will animate the frames (red, blue, and green cylinders) of the bones.
+
+## Problem 3 (Linear Blend Skinning)
+
+We now have the 3D transformation of each bone, let's animate the mesh using the linear blend skinning.
+
+Write some code around line #??? to implement the linear blend skinning.
+
+Each vertex of the mesh is associated with four bones.
+Use `inverseBindingMatrix` which has the inverse transformation of bone from origin to the undeformed mesh.
+
+This will animate black mesh.
+
+## After Doing the Assignment
+
+After modify the code, push the code and submit a pull request. Make sure your pull request only contains the files you edited. Good luck!
+
+BTW, You can exit the virtual environment by typing `deactivate` in the command prompt.
+
diff --git a/task08/main.py b/task08/main.py
new file mode 100644
index 0000000..b1de4e3
--- /dev/null
+++ b/task08/main.py
@@ -0,0 +1,175 @@
+import os
+import math
+#
+import pyrr
+import numpy as np
+import moderngl
+import moderngl_window as mglw
+from PIL import Image, ImageOps
+#
+import parse_gltf
+import util_for_task08
+
+
+
+class HelloWorld(mglw.WindowConfig):
+ '''
+ Window to show the gltf animation
+ '''
+ gl_version = (3, 3)
+ title = "task08: skeletal animation"
+ window_size = (500, 500)
+ aspect_ratio = float(window_size[0]) / float(window_size[1])
+ resizable = False
+
+ def __init__(self, **kwargs):
+ super().__init__(**kwargs)
+
+ asset_dir = os.path.normpath(os.path.join(__file__, '../../asset'))
+ tri2vtx, self.vtx2xyz_ini, self.vtx2bones, self.vtx2weights, \
+ self.bone2parentbone, self.bone2invBindingMatrix, \
+ self.channels = parse_gltf.load_data(
+ os.path.join(asset_dir, 'CesiumMan.gltf'),
+ os.path.join(asset_dir, 'CesiumMan_data.bin'))
+ tri2vtx = tri2vtx.astype(np.uint32).reshape(-1, 3)
+ edge2vtx = util_for_task08.edge2vtx_from_tri2vtx(tri2vtx) # extract edges from triangle mesh
+ self.vtx2xyz_def = self.vtx2xyz_ini.copy()
+ self.animation_duration = 0
+ for ch in self.channels:
+ self.animation_duration = max(self.animation_duration, ch.times.max())
+ self. is_screenshot_taken = False
+
+ self.prog = self.ctx.program(
+ vertex_shader='''
+ #version 330
+ uniform mat4 matrix;
+ in vec3 in_vert;
+ void main() {
+ gl_Position = matrix * vec4(in_vert, 1.0);
+ }
+ ''',
+ fragment_shader='''
+ #version 330
+ uniform vec3 color;
+ out vec4 f_color;
+ void main() {
+ f_color = vec4(color, 1.0);
+ }
+ ''',
+ )
+
+ ebo = self.ctx.buffer(edge2vtx) # send triangle index data to GPU (element buffer object)
+ vbo_ini = self.ctx.buffer(self.vtx2xyz_ini) # send initial vertex coordinates data to GPU
+ self.vao_ini = self.ctx.vertex_array(
+ self.prog, [(vbo_ini, '3f', 'in_vert')],
+ ebo, 4, mode=moderngl.LINES) # tell gpu about the mesh information and how to draw it
+ self.vbo_def = self.ctx.buffer(self.vtx2xyz_def) # send deformed vertex coordinates data to GPU
+ self.vao_def = self.ctx.vertex_array(
+ self.prog, [(self.vbo_def, '3f', 'in_vert')],
+ ebo, 4, mode=moderngl.LINES) # tell gpu about the mesh information and how to draw it
+
+ # cylinder mesh
+ (tri2vtx_cyl, vtx2xyz_cyl) = util_for_task08.cylinder_mesh_zup(0.01, 0.05, 16)
+ ebo = self.ctx.buffer(tri2vtx_cyl) # send triangle index data to GPU (element buffer object)
+ vbo_cyl = self.ctx.buffer(vtx2xyz_cyl) # send deformed vertex coordinates data to GPU
+ self.vao_cyl = self.ctx.vertex_array(
+ self.prog, [(vbo_cyl, '3f', 'in_vert')],
+ ebo, 4, mode=moderngl.TRIANGLES)
+
+ def render(self, time, frame_time):
+ time = time % self.animation_duration
+ num_bone = self.bone2invBindingMatrix.shape[0]
+
+ # bone2relativeTransformation is a numpy array with shape (#num_bone, 4, 4)
+ # bone2relativeTransformation[i_bone] is a matrix representing 3D transformation from the parent bone of i_bone
+ bone2relativeTransformation = parse_gltf.get_relative_transformations(time, num_bone, self.channels)
+
+ # bone2globalTransformation is a numpy array with shape (#num_bone, 4, 4)
+ # bone2globalTransformation[i_bone] is a matrix representing 3D transformation of each bone from the origin
+ bone2globalTransformation = np.zeros((num_bone, 4, 4))
+ for i_bone in range(num_bone):
+ i_bone_parent = self.bone2parentbone[i_bone]
+ if i_bone_parent == -1: # root bone
+ bone2globalTransformation[i_bone] = bone2relativeTransformation[i_bone]
+ continue
+ # below, write one or two lines of code to compute `bone2globalTransformation[i_bone]`
+ # hint: use numpy.matmul for multiplying nd-array
+ # bone2globalTransformation[i_bone] = ???
+
+ for i_vtx in range(self.vtx2xyz_ini.shape[0]): # for each point in mesh
+ p0 = self.vtx2xyz_ini[i_vtx]
+ p0 = np.append(p0, np.array([1.0])) # homogeneous coordinate of undeformed point
+ p1 = np.array([0., 0., 0., 1.], dtype=np.float32) # p1 is the deformed point
+ for idx in range(4): # in gltf each vertex is associated with four bones
+ w = self.vtx2weights[i_vtx][idx] # rig weight
+ i_bone = self.vtx2bones[i_vtx][idx] # bone index
+ inverseBindingMatrix = self.bone2invBindingMatrix[i_bone]
+ globalTransformation = bone2globalTransformation[i_bone]
+ # write a few lines of codes to compute p1 using the linear blend skinning
+ # hint: use np.matmul for matrix multiplication
+ # hint: assume that rig weights w add up to one
+
+ # p1 += ???
+
+ self.vtx2xyz_def[i_vtx] = p1[:3] # from homogeneous coordinates to the Cartesian coordinates
+
+ self.vbo_def.write(self.vtx2xyz_def)
+ self.ctx.clear(1.0, 1.0, 1.0)
+
+ # view transformation for undeformed character
+ transform_to_center = pyrr.Matrix44.from_translation((-0.8, 0., -0.8))
+ view_rot_x = pyrr.Matrix44.from_x_rotation(np.pi * 0.5)
+ view_rot_y = pyrr.Matrix44.from_y_rotation(np.pi * 0.3)
+ view_transf = view_rot_y * view_rot_x * transform_to_center
+
+ # draw undeformed mesh in red
+ self.prog['matrix'].value = tuple(view_transf.flatten())
+ self.prog['color'].value = (1., 0., 0.)
+ self.vao_ini.render()
+
+ # view transformation for deformed character
+ transform_to_center = pyrr.Matrix44.from_translation((+0.6, 0., -0.8))
+ view_rot_x = pyrr.Matrix44.from_x_rotation(np.pi * 0.5)
+ view_rot_y = pyrr.Matrix44.from_y_rotation(np.pi * 0.3)
+ view_transf = view_rot_y * view_rot_x * transform_to_center
+
+ # draw how the origin of each bone is transformed
+ for i_bone in range(num_bone):
+ transf = bone2globalTransformation[i_bone]
+ transf = np.matmul(transf.transpose(), view_transf)
+ # z_axis
+ self.prog['matrix'].value = tuple(transf.flatten())
+ self.prog['color'].value = (0., 0.0, 0.8)
+ self.vao_cyl.render()
+ # x_axis
+ x_rot = pyrr.Matrix44.from_y_rotation(math.pi*0.5)
+ self.prog['matrix'].value = tuple((np.matmul(x_rot.transpose(), transf)).flatten())
+ self.prog['color'].value = (0.8, 0., 0.)
+ self.vao_cyl.render()
+ # y_axis
+ y_rot = pyrr.Matrix44.from_x_rotation(-math.pi*0.5)
+ self.prog['matrix'].value = tuple((np.matmul(y_rot.transpose(), transf)).flatten())
+ self.prog['color'].value = (0.0, 0.8, 0.)
+ self.vao_cyl.render()
+
+ # draw deformed mesh in black
+ self.prog['matrix'].value = tuple(view_transf.flatten())
+ self.prog['color'].value = (0., 0., 0.)
+ self.vao_def.render()
+
+ # take a screenshot
+ if not self.is_screenshot_taken and time > 1.8:
+ self.is_screenshot_taken = True
+ rgb = np.frombuffer(self.ctx.fbo.read(), dtype=np.uint8)
+ rgb = rgb.reshape(self.ctx.fbo.size[0], self.ctx.fbo.size[1], 3)
+ rgb = Image.fromarray(rgb)
+ ImageOps.flip(rgb).save("output.png")
+
+
+
+def main():
+ HelloWorld.run()
+
+
+if __name__ == "__main__":
+ main()
diff --git a/task08/output.png b/task08/output.png
new file mode 100644
index 0000000..da031b9
Binary files /dev/null and b/task08/output.png differ
diff --git a/task08/parse_gltf.py b/task08/parse_gltf.py
new file mode 100644
index 0000000..e5c8650
--- /dev/null
+++ b/task08/parse_gltf.py
@@ -0,0 +1,149 @@
+from pathlib import Path
+import json
+#
+import numpy
+import pyrr
+
+
+class ChannelData:
+
+ def __init__(self, i_bone, times, values, path):
+ self.i_bone = i_bone
+ self.times = times
+ self.values = values
+ self.path = path # translation, scale, rotation
+
+ def get_value(self, time):
+ num_time = len(self.times)
+ for idx1 in range(num_time):
+ if time < self.times[idx1]:
+ break
+ if idx1 == num_time:
+ time = num_time - 1
+ idx0 = idx1 - 1
+ if idx0 == -1:
+ idx0 = 0
+ ratio = 0.0
+ if idx1 != idx0:
+ ratio = (time - self.times[idx0]) / (self.times[idx1] - self.times[idx0])
+ return (1. - ratio) * self.values[idx0] + ratio * self.values[idx1]
+
+
+def nparray_from_accessor(gltf, data, i_accessor):
+ accessor = gltf['accessors'][i_accessor]
+ #
+ dtype = numpy.dtype('float32')
+ if accessor['componentType'] == 5123:
+ dtype = numpy.dtype('ushort')
+ elif accessor['componentType'] == 5126:
+ dtype = numpy.dtype('float32')
+ #
+ size = 1
+ if accessor['type'] == 'VEC3':
+ size = 3
+ elif accessor['type'] == 'VEC4':
+ size = 4
+ elif accessor['type'] == 'MAT4':
+ size = 16
+ #
+ n = accessor['count']
+ bufferview = gltf['bufferViews'][accessor['bufferView']]
+ i_start = bufferview['byteOffset'] + accessor['byteOffset']
+ data_for_accessor = data[i_start:i_start + n * dtype.itemsize * size]
+ out_data = numpy.frombuffer(data_for_accessor, dtype=dtype)
+ #
+ if accessor['type'] == 'VEC3':
+ out_data = out_data.reshape(-1, 3)
+ elif accessor['type'] == 'VEC4':
+ out_data = out_data.reshape(-1, 4)
+ elif accessor['type'] == 'MAT4':
+ out_data = out_data.reshape(-1, 4, 4)
+ #
+ return out_data
+
+
+def get_bones_from_gltf(gltf, inode0, i_bone_parent, node2bone, bone2parentbone):
+ node = gltf['nodes'][inode0]
+ i_bone0 = node2bone[inode0]
+ bone2parentbone[i_bone0] = i_bone_parent
+ if not 'children' in node:
+ return
+ for i_node_chilren in node['children']:
+ get_bones_from_gltf(gltf, i_node_chilren, i_bone0, node2bone, bone2parentbone)
+
+
+def load_data(path_gltf, path_bin):
+ '''
+ load gltf data
+ :param path_gltf: file path for gltf
+ :param path_bin: file path for binary data (coordinates, animation etc)
+ :return: mesh data, blend skinning data, skeleton data, animation data
+ '''
+ file = open(path_gltf, 'r')
+ gltf = json.load(file)
+ data = Path(path_bin).read_bytes()
+
+ i_acc_tri2vtx = gltf['meshes'][0]['primitives'][0]['indices']
+ tri2vtx = nparray_from_accessor(gltf, data, i_acc_tri2vtx)
+ #
+ i_acc_vtx2xyz = gltf['meshes'][0]['primitives'][0]['attributes']['POSITION']
+ vtx2xyz = nparray_from_accessor(gltf, data, i_acc_vtx2xyz)
+ #
+ i_acc_vtx2weights = gltf['meshes'][0]['primitives'][0]['attributes']['WEIGHTS_0']
+ vtx2weights = nparray_from_accessor(gltf, data, i_acc_vtx2weights)
+ #
+ i_acc_vtx2joints = gltf['meshes'][0]['primitives'][0]['attributes']['JOINTS_0']
+ vtx2joints = nparray_from_accessor(gltf, data, i_acc_vtx2joints)
+ print(vtx2joints.shape)
+ #
+ for node in gltf['nodes']:
+ print(node)
+
+ print("******")
+
+ i_acc_bone2invBindingMatrix = gltf['skins'][0]['inverseBindMatrices']
+ bone2invBindingMatrix = nparray_from_accessor(gltf, data, i_acc_bone2invBindingMatrix).copy()
+ for i_bone in range(bone2invBindingMatrix.shape[0]):
+ bone2invBindingMatrix[i_bone] = bone2invBindingMatrix[i_bone].transpose()
+ print("bone2invBindingMatrix", bone2invBindingMatrix.shape)
+
+ bone2node = gltf['skins'][0]['joints']
+ node2bone = dict((v, k) for k, v in dict(enumerate(bone2node)).items())
+ # print(node2bone)
+
+ bone2boneparent = [-1] * len(bone2node)
+ i_node_skeleton_root = gltf['skins'][0]['skeleton']
+ get_bones_from_gltf(gltf, i_node_skeleton_root, -1, node2bone, bone2boneparent)
+ print(bone2boneparent)
+
+ channels = []
+ for channel in gltf['animations'][0]['channels']:
+ path = channel['target']['path'] # rotation, scale target
+ i_bone = node2bone[channel['target']['node']]
+ sampler = gltf['animations'][0]['samplers'][channel['sampler']]
+ assert (sampler['interpolation'] == 'LINEAR')
+ i_acc_times = sampler['input']
+ i_acc_values = sampler['output']
+ value_times = nparray_from_accessor(gltf, data, i_acc_times)
+ value_values = nparray_from_accessor(gltf, data, i_acc_values)
+ channels.append(ChannelData(i_bone, value_times, value_values, path))
+
+ return tri2vtx, vtx2xyz, vtx2joints, vtx2weights, bone2boneparent, bone2invBindingMatrix, channels
+
+
+def get_relative_transformations(time: float, num_bone: int, channels):
+ bone2relativeTransformation = numpy.zeros((num_bone, 4, 4))
+ for i_bone in range(num_bone):
+ bone2relativeTransformation[i_bone] = numpy.eye(4)
+ for ch in channels:
+ val = ch.get_value(time)
+ i_bone = ch.i_bone
+ if ch.path == "translation":
+ m = pyrr.Matrix44.from_translation(val).transpose()
+ bone2relativeTransformation[i_bone] = numpy.matmul(bone2relativeTransformation[i_bone], m)
+ elif ch.path == "rotation":
+ m = pyrr.Matrix44.from_quaternion(val)
+ bone2relativeTransformation[i_bone] = numpy.matmul(bone2relativeTransformation[i_bone], m)
+ else:
+ pass
+ return bone2relativeTransformation
diff --git a/task08/preview.png b/task08/preview.png
index 2a86dfb..0e943a8 100644
Binary files a/task08/preview.png and b/task08/preview.png differ
diff --git a/task08/requirements.txt b/task08/requirements.txt
new file mode 100644
index 0000000..52d7567
--- /dev/null
+++ b/task08/requirements.txt
@@ -0,0 +1,8 @@
+glcontext==2.5.0
+moderngl==5.10.0
+moderngl-window==2.4.6
+multipledispatch==1.0.0
+numpy==1.26.4
+pillow==10.3.0
+pyglet==2.0.15
+pyrr==0.10.3
diff --git a/task08/util_for_task08.py b/task08/util_for_task08.py
new file mode 100644
index 0000000..b0d7edd
--- /dev/null
+++ b/task08/util_for_task08.py
@@ -0,0 +1,55 @@
+import numpy
+import numpy.typing
+import math
+
+def edge2vtx_from_tri2vtx(tri2vtx) -> numpy.typing.NDArray:
+ '''
+ :param tri2vtx: triangle index of a mesh
+ :return: edges in the triangle mesh (shape=[#edge,2], dtype=np.int32)
+ '''
+ v0 = tri2vtx[:, 0]
+ v1 = tri2vtx[:, 1]
+ v2 = tri2vtx[:, 2]
+ edge2vtx = numpy.stack([v0, v1], axis=1)
+ edge2vtx = numpy.vstack([edge2vtx, numpy.stack([v1, v2], axis=1)])
+ edge2vtx = numpy.vstack([edge2vtx, numpy.stack([v2, v0], axis=1)])
+ edge2vtx.sort(axis=1)
+ edge2vtx = numpy.unique(edge2vtx, axis=0)
+ return edge2vtx
+
+
+def cylinder_mesh_zup(r, l, n) -> (numpy.typing.NDArray, numpy.typing.NDArray):
+ '''
+
+ :param r:
+ :param l:
+ :param n:
+ :return: triangle index and vertex coordinates
+ '''
+ tri2vtx = numpy.zeros((4*n, 3), dtype=numpy.uint32)
+ for i in range(0, n):
+ tri2vtx[i, 0] = 0
+ tri2vtx[i, 1] = 1+i
+ tri2vtx[i, 2] = 1+(i+1)%n
+ #
+ tri2vtx[n+i, 0] = 1+i
+ tri2vtx[n+i, 1] = 1+(i+1)%n
+ tri2vtx[n+i, 2] = 1+(i+1)%n+n
+ #
+ tri2vtx[2*n+i, 0] = 1+i
+ tri2vtx[2*n+i, 1] = 1+(i+1)%n+n
+ tri2vtx[2*n+i, 2] = 1+i+n
+ #
+ tri2vtx[3*n+i, 0] = 2*n+1
+ tri2vtx[3*n+i, 1] = 1+(i+1)%n+n
+ tri2vtx[3*n+i, 2] = 1+i+n
+
+ vtx2xyz = numpy.zeros((2*n+2, 3), dtype=numpy.float32)
+ vtx2xyz[0, :] = [0., 0., 0.]
+ for i in range(0, n):
+ theta = math.pi * 2.0 * float(i) / float(n)
+ vtx2xyz[1+i, :] = [r*math.cos(theta), r*math.sin(theta), 0.]
+ vtx2xyz[1+i+n, :] = [r*math.cos(theta), r*math.sin(theta), l]
+ vtx2xyz[2*n+1, :] = [0., 0., l]
+
+ return (tri2vtx, vtx2xyz)