Optimization of parameter settings for GAMG solver in simple solver, OpenFOAM Study Meeting for beginner@Kanto, 8th Sep. 2012
12 likes•9,351 views
The document summarizes presentations given by Masashi Imano of OCAEL Co. Ltd. at OpenFOAM study meetings for beginners in Kansai and Kanto, Japan. It discusses optimizing parameters for the GAMG solver in OpenFOAM, including the number of cells in the coarsest grid level. Testing on a 16-node SGI cluster showed the optimal range was 32-1024 cells. It also discusses parameters like merge levels, number of smoothing sweeps, and their effect on solver speed for different node counts. The document provides guidance on selecting parameters for the GAMG solver in OpenFOAM simulations.
Optimization of parameter settings for GAMG solver in simple solver, OpenFOAM Study Meeting for beginner@Kanto, 8th Sep. 2012
- 1. Aug. 26th 2012 OpenFOAM Study Meeting for beginner @ Kanto Masashi Imano (OCAEL Co. Ltd.) Masashi Imano (OCAEL Co.Ltd.) Optimization of parameter settings for GAMG solver in simple solver
- 2. Aug. 8th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) Test cluster condition • Hardware: SGI Altix ICE8200 • CPU: Intel Xeon X5365 3.00GHz 4cores x 2 • Node: 16 nodes • Memory: 16GB/node (No swap memory) • Interconnect: InfiniBand DDR • OS: SUSE LINUX ENTERPRISE SERVER 10.3 • MPI: ✓ SGI MPT (1.2.6)
- 3. Aug. 26th 2012 OpenFOAM Study Meeting for beginner @ Kanto Masashi Imano (OCAEL Co. Ltd.) “Guidebook for Practical Applications of CFD to Pedestrian Wind Environment around Buildings” Working Group in Architectural Institute of Japan Web site (in Japanese and English)( in Japanese) Akashi Mochida, Yoshihide Tominaga, et al. Guideline for Practical Applications
- 4. Aug. 26th 2012 OpenFOAM Study Meeting for beginner @ Kanto Masashi Imano (OCAEL Co. Ltd.) Case A (2:1:1 shape) Case B (4:4:1 shape) Case C (Simple blocks) Case D (High-rise bldg.) Case E (Niigata) Case F (Shinjuku, Tokyo) Benchmark tests
- 5. Aug. 26th 2012 OpenFOAM Study Meeting for beginner @ Kanto Masashi Imano (OCAEL Co. Ltd.) Mesh Cartesian mesh (snappyHexMesh) 2.2 millions Inflow Interpolate from wind tunnel results Top & Side wall Zero gradient Ground and bldg. wall Generalized log law for a smooth wall Turbulence model Standard k-epsilon Advection scheme Upwind Algorithm SIMPLE (simpleFoam) Wind direction NNE Building complexes with simple building shape in actual urban area (Niigata) Case E - Calculation condition
- 6. Aug. 8th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) Accelerate incompressible solver • CPU time of linear solver for pressure is dominant: ✓ Many nodes : PCG is faster ✓ Not so many nodes : GAMG is faster ➡ Need to optimize of parameter for GAMG T2K at The University of Tokyo Node: 16 cores Interconnect: Myrinet 2.5GB/s X 2 Solver: ChannelFoam Mesh: 256x256x256 Pre-conditioner for GAMG: DIC Better Worse PCG is faster GAMG is faster
- 7. Aug. 8th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) GAMG Geometric agglomerated algebraic multi-grid solver (Generalised geometric-algebraic multi-grid in Userguide) http://www.lrr.in.tum.de/Par/appls/apps/amg.html
- 8. Aug. 8th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) Geometric agglomeration process Tim Behrens: OpenFOAM's basic solvers for linear systems of equations Solvers, preconditioners, smoothers, PhD course in CFD with OpenSource software, Technical University of Denmark, February 18, 2009 Merge Finest grid (Lv.1) Coarse grid (Lv.2) agglomerator faceAreaPair; mergeLevels 1; Coarsest grid (Lv.3)
- 9. Aug. 8th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) Parameter settings for AMG solver smoother GaussSeidel, DIC, DICGaussSeidel; // GaussSeidel is usually best directSolveCoarsest false, true; // false: Use ICCG/BICCG for coarsest grid, true: Solve directly in coarsest grid agglomerator faceAreaPair, algebraicPair; // faceAreaPair is usually better than algebraicPair mergeLevels 1, 2, 3, ..; // Usually 1 is better. For simple meshes setting 2 could be better. nPreSweeps 0, 1, 2, ..; // Number of pre-smoothing sweeps nPostSweeps 0, 1, 2, ..; // Number of post-smoothing sweeps nFinestSweeps 0, 1, 2,..; // Number of smoothing sweeps on finest mesh nCellsInCoarsestLevel 1, 2, ..; // Number of cells in coarsest level mesh system/fvSolution:
- 10. Aug. 8th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) Study about nCellsInCoarsestLevel smoother GaussSeidel, DIC, DICGaussSeidel; // GaussSeidel is usually best directSolveCoarsest false, true; // true: Use ICCG/BICCG for coarsest grid, false: Solve directly in coarsest grid agglomerator faceAreaPair, algebraicPair; // faceAreaPair is usually better than algebraicPair mergeLevels 1, 2, 3, ..; // Usually 1 is better. For simple meshes setting 2 could be better. nPreSweeps 0, 1, 2, 3, ..; // Number of pre-smoothing sweeps nPostSweeps 0, 1, 2, 3, ..; // Number of post-smoothing sweeps nFinestSweeps 0, 1, 2, 3,..; // Number of smoothing sweeps on finest mesh nCellsInCoarsestLevel 8, 16, 32, .., 256, .., 2048, 4096, 8192; // Number of cells in coarsest level mesh system/fvSolution:
- 11. Aug. 8th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) Other calculation conditions relaxationFactors // 0 < relaxationFactors <= 1 { p 0.3; // for pressure (important) U 0.5; // for velocity vectors (important) k 0.5; // for turbulence kinetic energy epsilon 0.5; // for turbulence kinetit energy // dissipation rate } system/fvSolution: Solver: simpleFoam Iitial field conditions: potentialFoam CPU time: 50 steps (No write result field)
- 12. Aug. 8th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) Results of speedup ratio Speedupratio(base256cells) 0.7 0.8 0.9 1 1.1 Number of Cells In Coarsest Level Grid 8 16 32 64 128 256 512 1024 2048 4096 8192 1 node 2 nodes 4 nodes 8 nodes 16 nodes Better range for any nodes Worse Better
- 13. Aug. 8th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) Study about nCellsInCoarsestLevel smoother GaussSeidel, DIC, DICGaussSeidel; // GaussSeidel is usually best directSolveCoarsest false, true; // true: Use ICCG/BICCG for coarsest grid, false: Solve directly in coarsest grid agglomerator faceAreaPair, algebraicPair; // faceAreaPair is usually better than algebraicPair mergeLevels 1, 2, 3, ..; // Usually 1 is better. For simple meshes setting 2 could be better. nPreSweeps 0, 1, 2, 3, ..; // Number of pre-smoothing sweeps nPostSweeps 0, 1, 2, 3, ..; // Number of post-smoothing sweeps nFinestSweeps 0, 1, 2, 3,..; // Number of smoothing sweeps on finest mesh nCellsInCoarsestLevel 8, 16, 32, .., 256, .., 2048, 4096, 8192; // Number of cells in coarsest level mesh system/fvSolution:
- 14. Aug. 8th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) Results of speedup ratio Speedupratio(basesetting) 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1 node 2 nodes 4 nodes 8 nodes 16 nodes Worse Better mergeLevels 1 2 nPreSweeps 0 1 0 1 nPostSweeps 1 2 3 1 2 3 1 2 3 1 2 3 nFinestSwee 2 3 2 3 2 3 2 3 2 3 2 3 2 3 2 3 2 3 2 3 2 3 2 3 Better range for any nodes Base setting 16 nodes (Speedup ratio could decrease in many node parallel calculation when mergeLevels is 2)
- 15. Aug. 26th 2012 OpenFOAM Study Meeting for beginner @ Kanto Masashi Imano (OCAEL Co. Ltd.) Any Questions?