Computer Graphics Recipes for Endowing 3D Shapes with Semantics in Virtual Worlds
1 like811 views
The document discusses the integration of semantics into 3D graphics to enhance virtual worlds by making shapes more meaningful and understandable. It outlines methods for extracting and associating knowledge with digital content, focusing on shape analysis, segmentation, and annotation. The potential applications include semantic search for model reuse and simulations in assistive living, while highlighting the challenges of manual annotation and the need for a shared conceptualization.
Computer Graphics Recipes for Endowing 3D Shapes with Semantics in Virtual Worlds
- 1. Computer Graphics Recipes for Endowing 3D Shapes with Semantics in Virtual Worlds Michela Mortara, Chiara E. Catalano CNR IMATI Genova, Italy
- 2. Outline 3D in Gaming: need of semantics How to make semantics explicit: Segmentation Annotation Applications to virtual worlds Conclusions 3AMIGAS Workshop, Amstedam, June 16th 2009
- 3. 3D in Gaming Complex and convincing virtual worlds realistic environment and characters realistic interactions and behaviour 3AMIGAS Workshop, Amstedam, June 16th 2009
- 4. What’s semantics? Knowledge related to the geometry of the 3D shape Knowledge related to the application domain in which the object is manipulated Knowledge related to the meaning of the object represented by the 3D model 3AMIGAS Workshop, Amstedam, June 16th 2009
- 5. Adding semantics to geometry We have to extract and associate knowledge to digital content Shape analysis: extracts implicit info coded into the geometry Numerical values (e.g. volume, area, spatial distribution, curvature) Shape descriptors (structuring the shape) Segmentation 3AMIGAS Workshop, Amstedam, June 16th 2009 geometry structure semantics
- 6. Adding semantics to geometry We have to extract and associate knowledge to digital content and its subparts Shape Annotation: it documents the shape with contextual knowledge either manually or automatically geometry structure semantics 3AMIGAS Workshop, Amstedam, June 16th 2009
- 7. Semantic-driven segmentations They captures semantically relevant features automatically what is relevant in the gaming context? IMATI segmentation methods for triangle meshes: Fitting primitives Reeb Graphs Tailor Plumber They extract specific kind of knowledge suitable also for virtual worlds 3AMIGAS Workshop, Amstedam, June 16th 2009
- 8. Fitting Primitives Hierarchical face clustering algorithm able to recognise planes, cylinders and spheres 3AMIGAS Workshop, Amstedam, June 16th 2009
- 9. Reeb Graph The topology of a shape is coded into the Reeb graph respect to f, which captures the evolution of the level sets of f on the shape 3AMIGAS Workshop, Amstedam, June 16th 2009
- 10. Plumber (and Tailor) It defines a shape decomposition into connected components that are either tubular features or blob regions Skeletons and grasping regions 3AMIGAS Workshop, Amstedam, June 16th 2009 Landmarks
- 11. Automatic identification of human body parts Plumber Tailor Skeleton extraction segmentation characterisation Tagging human Candidate body parts skeletal joints 3AMIGAS Workshop, Amstedam, June 16th 2009
- 12. ShapeAnnotator: general framework Shape A surface mesh Abstracted Shape Shape A segmented mesh CG Tools Segmentation Plug-ins Annotator Domain An OWL ontology Instance Instance Expert Knowledge Base 3AMIGAS Workshop, Amstedam, June 16th 2009
- 13. ShapeAnnotator: multi-segmentation Pick the interesting features from different shape segmentations (b) Morse-based (c) Plumber (d) fitting primitives 3AMIGAS Workshop, Amstedam, June 16th 2009
- 14. The ShapeAnnotator: ontology browser 3AMIGAS Workshop, Amstedam, June 16th 2009
- 15. The ShapeAnnotator: knowledge base 3AMIGAS Workshop, Amstedam, June 16th 2009
- 16. Possible scenarios for virtual worlds DSW semantic search for reuse: Models as they are: to have large diversity of human characters and objects Modifying models: to create new VHs, reuse garments, take only interesting parts Assistive living Simulation of domestic environments with realistic people (e.g. with disabilities), places and danger conditions 3AMIGAS Workshop, Amstedam, June 16th 2009
- 17. Conclusions 3D with semantics would have high impact Easy indexing, searching and retrieval Efficient reuse and re-adaptation Realistic interactions among virtual characters and objects Still a lot of work to do Manual annotation No shared conceptualisation … 3AMIGAS Workshop, Amstedam, June 16th 2009