Download to read offline
More Related Content
Similar to Sharing with Europeana: Depositing and publishing 3D datasets for preservation and future access
Sharing with Europeana: Depositing and publishing 3D datasets for preservation and future access
- 1. Kate Fernie, CARARE Sharingwith Europeana: Depositing and publishing 3D datasets for preservation and future access
- 2. Introduction ● Twin iT!marks a massive expansion of 3D in the cultural heritage sector ● The webinars in this series by 4CH have covered the digitisation workflow from the initial plans to modelling techniques ● Today I will talk about the data that you create and strategies for depositing and publishing your results Oxbow Books, 15 Dec. 2003 ISBN: 1842170406
- 3. From fieldwork to depositingyour 3D dataset and providing access 3
- 4. 3D Workflow CAPTURE MODELLINGTEXTURING OUTPUTs Point Cloud s High Poly Model Low Poly Model Textures Photos
- 5. Workflow and data 3Dworkflow uses various equipment and software ● Survey equipment which produces raw data in various file types ● Software to process the data to produce the desired results which is capable of ○ accepting data in certain file types ○ exporting data in certain file types
- 6. From fieldwork to depositingyour 3D dataset and providing access Interoperability matters during the project and for future access https://open-science-training-handbook.github.io/Open-Science-Training-Handbook_EN/
- 7. What’s in a3D dataset? 7
- 8. Laser scanning Thousands ofpoints representing X, Y, Z coordinates, colour and intensity Multiple scans Imported into software for registration and processing Photogrammetry Thousands of pictures taken at different angles around a CH object Imported into software which uses a Structure from Motion (SfM) algorithm to produce a point cloud or a mesh Reference measures GNSS or GPS ground control points TPS points on building features and elements Used for registering laser scan data and photogrammetry Data capture
- 9. Data - initialprocessing ● Data produced in 3D projects is rarely used in raw, unedited form ● In most projects there will be multiple datasets (e.g. overview and detail scans, photogrammetry, drone imagery, etc.) which are each registered to a coordinate system to create a unified point cloud ● Point clouds are used to generate various products (such as polygonal meshes, CAD drawings and elevation models)
- 10. File types/formats A lotof different hardware and software is involved. There are many file formats but common file types have potential for deposit and archiving. Photogrammetry Import - images (proprietary raw or standard JPG/TIFF) Export - point cloud/, meshes and textures Point clouds Original raw scan data (proprietary format) + XYZ (ASCII text) Registered point cloud: ● XYZ (ASCII text) ● E57 (evolving as a standard) ● LAS ● proprietary format … Meshes https://archaeologydataservice.ac.uk/help-guidance/guides-to-good-practice/data-collection-and-fieldwork/ laser-scanning-for-archaeology/archiving-laser-scan-data/file-formats-for-archiving-datasets/
- 11. From data to3D model ● In reality capture the point cloud is used to generate a mesh which defines the shape of an object ● The data undergoes processing/editing ● Textures are applied and ● The mesh is optimized ● A high poly count mesh is the first output ○ from this you can generate lower polygon count meshes ○ models in various formats (according to the planned uses) 137 million polygons Discovery Programme
- 12. File types/formats Meshes Open formats: ●OBJ + MTL ● Collada (.dae) ● X3D Other formats: ● PLY ● glTF ● STL (for printing) Textures Can be saved as: ● JPEG ● PNG ● TIFF ● TARGA (TGA)
- 13. Need for archiving Lowerpriority ● Derivatives - 3D models for web; for 3D printing etc. High priority ● Original raw scan data (proprietary & ASCII) ● Registered point cloud (ASCII + Open format) ● Original images (photogrammetry) ● High Poly mesh + textures ● Low poly mesh
- 14. Data files plusmetadata https://open-science-training-handbook.github.io/Open-Science-Training-Handbook_EN/ What’s in a 3D dataset?
- 15. Metadata When depositing 3Ddatasets archives need good documentation to enable preservation and access, this includes: ● Details about the project and how the content was produced ● The technical characteristics – tools and software used, data processing, control, geometry, textures, etc. ● Info to enable discovery and access – the CH subject, period, copyright, access conditions, media, links etc.
- 16. Depositing for future access SeanDoran, Discovery Programme
- 17. Plan for archivingfrom the start of the project Image: European Data Journalism Network Find a suitable digital archive or repository and check their requirements and what level of access they can offer. ● University repositories ● (larger) cultural institutional repositories ● National repositories such as DANS Data Vault, SND and DRI ● International repositories such as https://zenodo.org/ ● Specialist data archives such as ADS
- 18. Preparing for deposit Don’tleave preparation of the dataset for archiving to the end of the project ● Specify the data for archiving ● Specify formats which have most potential for archiving ○ open, industry standard ● Specify the documentation and metadata that’s needed ● Agree the copyright status and access permissions
- 19. Deposit the datasetat the end of the project https://doi.org10.5281/zenodo.77350786 Now you can plan to publish other outputs!
- 20. The 3D workflowcan produce many outputs – from high poly models for HBIM, low poly models for publishing online to 3D models for printing and more. These can be uploaded with texture files for publication on hosting platforms: ● Commercial ● National ● Local Publishing
- 21. Viewers Ideally you wouldselect a hosting platform that can render your 3D model in a viewers. ● There are 3D viewers available as open source software which can be installed on your local server/repository. For example, 3D Hop and the Smithsonian’s 3D viewer. ● The alternative is to upload the content to a hosting platform (commercial or non-commercial) and take advantage of their viewer
- 22. Share3D I’ve spoken previously aboutShare3D, which aims to help CH Sketchfab users to share their 3D models with Europeana. Sign up for an account here: https://dashboard.share3 d.eu/login.php
- 23. More publishing options SmithsonianVoyager https://smithsonian.github.io/dpo-voyager/
- 24. ● EC directivefocusses on cultural heritage at risk and the most visited sites ● 3D digitisation enables monitoring of the heritage (changes over time) and increases access ● Securing your results for the future ● Increases the possibility for reuse ● Leads to future research ● Supports tourism, education, enjoyment Why storing and preserving your results matters Monitoring
- 25. Acknowledgements and references AnthonyCorns, Technology Manager, Discovery Programme Marco Medici, INCEPTION 4CH Deliverable D3.1 Design of the CH Cloud and 4CH platform 4CH Deliverable D4.1 Report on standards, procedures and protocols ADS Guides to Good Practice 25
- 26. Kate Fernie [email protected] Thanks foryour attention! https://www.carare.eu Training hub: https://pro.carare.eu/en/training-hub/ CARARE is a non-profit membership association which offers advice, guidance and training in the creation, publication and use of digital data. A Europeana aggregator CARARE offers technical services to help institutions share their archaeological and architectural heritage content.
Editor's Notes
- #2 Preservation is a challenging topic even if 3D has been around for quite a while.
- #4 At every stage in the 3D workflow you are creating data, integrating data from different sources and processing it to produce various outputs.
- #8 GPS Global Positioning System TPS Terrestrial Positioning System
- #10 A lot of different hardware and software is involved which doesn’t all play well together. Most software has file export and import options. There are many file types but among them there are some common types which have potential for archiving. Photogrammetry begins by collecting a lot of images, which are imported either as camera raw files (proprietary) or as standard JPGs. Standard archival recommendations for image files apply. The images are used to generate a point cloud or a mesh and textures. These are usually output from photogrammetry software and there is generally a choice of file formats for export, choosing open file formats (discussed next) is recommended. Scanner manufacturers have their own software for processing the point clouds generated; the individual scans are registered and brought together to create a unified point cloud. The data can be exported in the proprietary format, or in a format such as XYZ (an ASCII text file), E57 or LAS. XYZ, E57 and LAS are not yet formal standards but they are on a promising track and are being widely adopted by manufacturers. For archiving the recommendation is to deposit the original raw scan data in both the original proprietary format and as ASCII text (XYZ); plus the registered point cloud - again as ASCII text plus proprietary / E57 and/or LAS. Multiple formats are recommended for archiving as these datasets undergo considerable editing/processing, which can potentially remove valuable data. The ASCII version of a scan is good for long term preservation but it may look different to the version in the proprietary format or a format such as E57. This is a more is more approach to a challenging preservation scenario. ASTM E57 is a promising option as a standard for laser scan data as it is supported by a number of vendors, scanners and systems. Files can store 3D point data with attributes (colour and intensity) and also 2D imagery obtained from the system. As well as being used in laser scanners, E57 can be used in flash LIDAR systems, structured light scanners and other systems. The LAS format was developed by ASPRS to store aerial LIDAR data but is gaining support in the terrestrial scanning community.
- #11 At this stage the point cloud is ingested into software and a polygon mesh is generated - a mesh is basically a collection of triangles or quadrilateral vertices which define the shape of the object. The data undergoes a lot of processing as the mesh is produced and worked on. The data is cleaned up - noise removed, any holes (where no points were captured) are filled, unwanted objects are removed (survey targets, vegetation) - optimized and textures are applied. The first step results in a high polygon count mesh. You will most likely also generate lower polygon count models (according to the requirements of your project). High polygon count models are slow to render and so for online viewing you will generally need to decimate the data to produce lower poly models.
- #19 https://about.zenodo.org/ Zenodo accepts datasets from EC funded projects as well as research institutions.