Accuracy, Convergence and Mesh Quality

[This article was first published in the May/June 2012 issue of The Connector. It was so popular there it's reposted here to reach a broader audience.]

“We know embarrassingly little about how the mesh affects the CFD solution,” said Prof. Carl Ollivier-Gooch of the University of British Columbia.

That statement is counter to what we all know to be true in practice, that a good mesh helps the computational fluid dynamics (CFD) solver converge to the correct answer while minimizing the computer resources expended. Stated differently, most every decent solver will yield an accurate answer with a good mesh, but it takes the most robust of solvers to get an answer on a bad mesh.

The crux of the issue is what precisely is meant by “a good mesh.” Syracuse University’s Prof. John Dannenhoffer points out that we are much better at identifying a bad mesh than we are at judging a good one. Distinguishing good from bad is clouded by the fact that badness is a black-white determination of whether the mesh will run or not. (Badness often only means whether there are any negative volume cells.) On the other hand, goodness is all shades of gray – there are good meshes and there are better meshes.

Neither is goodness all about the mesh. Gone are the days when one could eyeball the mesh and make a good/bad judgment. Adaptive meshes that are justified by visual inspection of how much thinner shock waves are in a contour plot of density just do not make the grade. What matters is how accurately the CFD solution reflects reality. Therefore, the solver’s numerical algorithm and the physics of the flow to be computed also have to be accounted for in the evaluation of a mesh.

Implicit in the paragraphs above is the idea of judging mesh quality in advance of computing the CFD solution. There are those who think that a priori mesh quality assessment is of limited value and that changing the mesh in response to the developing flow solution (via mesh adaption or adjoint methods or other technology) is the better way to generate a good mesh and an accurate solution.

Mesh Quality Workshop

Given this state of affairs, it was important to assemble mesh generation researchers and practitioners to assess the topic of mesh quality. Pointwise participated in the “Mesh Quality/Resolution, Practice, Current Research, and Future Directions Workshop” last summer in Dayton and hosted by the DoD High Performance Computing Modernization Program (HPCMO) and organized by the PETTT Program (User Productivity, Enhancement, Technology Transfer and Training) and AIAA’s MVCE Technical Committee (Meshing, Visualization, and Computational Environments).

The workshop brought together all the stakeholders of mesh quality: CFD practitioners, CFD researchers, CFD solver code developers (both commercial and government) and mesh generation software developers. A list of the workshop presentations is included at the end of this article (References 1a-1i). Hugh Thornburg from High Performance Technologies wrote an overview of the workshop (Reference 2) that nicely sums up the current state of affairs:

  • “A mesh as an intermediate product has no inherent requirements and only needs to be sufficient to facilitate the prediction of the desired result.” I interpret this as the double-negative quality judgment that the grid is “not bad.”
  • “The mesh must capture the system/problem of interest in a discrete manner with sufficient detail to enable the desired simulation to be performed.” As long as “desired simulation” implicitly includes “to a desired level of accuracy,” this is a good definition.
  • Thornburg also acknowledges many practical constraints on mesh generation such as time allotted for meshing, topology issues for parametric studies, limits on mesh size due to computational resources, and solver-specific requirements.

Thornburg also offers Simpson’s Verdict library (Reference 3) as a de facto reference that covers “most if not all commonly used techniques” for computing element properties.

User’s Perspective

The importance of a priori indicators of mesh quality is exemplified by NASA’s Stephen Alter, who defined and demonstrated the utility of his GQ (grid quality) metric that combines both orthogonality and stretching into a single number. Driven by the desire to ensure the accuracy of supersonic flow solutions over blunt bodies computed using a thin layer Navier-Stokes solver, he has established criteria for the GQ metric that give him confidence prior to starting a CFD solution.

Two aspects of GQ are notable. First, this metric’s reliance on orthogonality is closely coupled to the numerics of the solver – TLNS assumptions break down when the grid lacks orthogonality. Second, use of a global metric aids decision making, or as Thornburg wrote, “A local error estimate is of little use.” GQ represents domain expertise – the use of specific criteria within a specific application domain.

Researcher’s Perspective

Dannenhoffer reported on an extensive benchmark study that involved parametric variation of a structured grid’s quality for a 5 degree double-wedge airfoil in Mach 2 inviscid flow at 3 degrees angle of attack. Variations of the mesh included resolution, aspect ratio, clustering, skew, taper, and wiggle (using the Verdict definitions).

Dannenhoffer’s main conclusion was very interesting: there was little (if any) correlation between the grid metrics and solution accuracy. This may have been exacerbated by the fact that he found it difficult to change one metric without influencing another (e.g. adding wiggle to the mesh also affected skew) or it may have been due to the specific flow conditions.

Dannenhoffer also introduced the concept of grid validity (as opposed to grid quality), which is intended to measure whether the grid conforms to the configuration being modeled (which in practice it sometimes does not). He proposed three types of validity checks:

  1. Type 1 checks whether cells have positive volumes and faces that do not intersect each other. Here again is an instance of the “Is this grid bad?” question.
  2. Type 2 checks whether interior cell faces match uniquely with one other interior face and whether boundary cell faces lie on the geometry model of the object being meshed.
  3. Type 3 checks whether each surface of the geometry model is completely covered by boundary cell faces, whether each hard edge of the geometry is covered by edges of boundary cell faces, and whether the sum of the boundary faces areas matches the actual geometry surface area.

Figure 1: A simple demonstration of how a poor mesh from a cell geometry perspective (right) results in lower discretization error than one with “perfect” cells (left). From Reference 1c.

Prof. Christopher Roy from Virginia Tech showed a counter-intuitive example (at least from the standpoint of a priori metrics) that the solution of 2D Burger’s equation on an adapted mesh (with cells of widely varying skew, aspect ratio, and other metrics) has much less discretization error than the solution on a mesh of perfect squares. From this example alone, it is clear that metrics based solely on cell geometry are not good indicators of mesh quality as it pertains to solution accuracy.

Solver’s Perspective

The workshop was fortunate to have the participation of several flow solver developers, who shared details about how their solver is affected by mesh quality. The common thread among all was that convergence and stability are more directly affected by mesh quality than solution accuracy.

CFD++

Metacomp Technologies’ Vinit Gupta cited cell skewness and cell size variation as two quality issues to be aware of for structured grids. In particular, grid refinement across block boundaries in the far field where gradients are low has a strong, negative impact on convergence. For unstructured and hybrid meshes, anisotropic tets in the boundary layer and the transition from prisms to tets outside the boundary layer also can be problematic.

Gupta also pointed out two problems associated with metric computations. Cell volume computations that rely on a decomposition of a cell into tets are not unique and depend on the manner of decomposition. Therefore, volume (or any measure that relies on volume) reported by one program may differ from that reported by another. Similarly, face normal computations for anything but a triangle are not unique and also may differ from program to program. (This is a scenario we have often encountered at Pointwise when there is a disagreement with a solver vendor over a cell’s volume that turns out to be the result of different computation methods.)

Fluent and CFX

ANSYS’ Konstantine Kourbatski showed how cell shapes that differ from perfect (dot product of face normal vector with vector connecting adjacent cell centers) make the system of equations stiffer slowing convergence.  He then introduced metrics, Orthogonal Quality and two skewness definitions, with rules of thumb for the Fluent solver. It was interesting to note that the orthogonality measure ranges from 0 (bad) to 1 (good) whereas the skewness metric is directly opposite: 0 is good and 1 is bad. Another example of a metric criterion was that aspect ratios should be kept to less than 5 in the bulk flow. Kourbatski also provided guidelines for the CFX solver.

He also pointed out that resolution of critical flow features (e.g. shear layers, shock waves) is vital to an accurate solution and that bad cells in benign flow regions usually do not have a significant effect on the solution.

Kestrel

Kestrel, the CFD solver from the CREATE-AV program, was represented by David McDaniel from the University of Alabama at Birmingham. At the start, he made two important statements. First, their goal is to “do well with the mesh given to us.” (This is similar to Pointwise’s approach to dealing with CAD geometry – do the absolute best with the geometry provided.) Second, he notes that mixed-element unstructured meshes (their primary type) are terrible according to traditional mesh metrics, despite being known to yield accurate results. This same observation is true for adaptive meshes and meshes distorted by the relative motion of bodies within a mesh (e.g. flaps deflecting, stores dropping).

More significantly, McDaniel notes a “scary” interdependence between solver discretization and mesh geometry by recalling Mavriplis’ paper on the drag prediction workshop (Reference 4) in which two extremely similar meshes yielded vastly different results with multiple solvers.

To address mesh quality, Kestrel’s developers have implemented non-dimensional quality metrics that are both local and global and that are consistent in the sense that 0 always means bad and 1 always means good. The metrics important to Kestrel are an area-weighted measure of quad face planarity, an interesting measure of flow alignment with the nearest solid boundary, a least squares gradient that accounts for the orientation and proximity of neighbor cell centroids, smoothness, spacing and isotropy.

Figure 2: Using Kestrel one can show a correlation between mesh and solution quality. From Reference 1f.

Differing from Dannenhoffer’s result, McDaniel showed a correlation of mesh quality with solution accuracy with the caveat that a well resolved mesh can have poor quality and still produce a good answer. (In other words, more points always is better.)

STAR-CCM+

Alan Mueller’s presentation on CD-adapco’s STAR-CCM+ solver began by pointing out that mesh quality begins with CAD geometry quality and manifests as either a low quality surface mesh or an inaccurate representation of the true shape. This echoes Dannenhoffer’s grid validity idea.

After introducing a list of their quality metrics, Mueller makes the following statement, “Results on less than perfect meshes are essentially the same (drag and lift) as on meshes where considerable resources were spent to eliminate the poor cells in the mesh.” Here we note that the objective functions are integrated quantities (drag and lift,) instead of distributed data like pressure profiles. After all, integrated quantities are the type of engineering data we want to get from CFD.

This insensitivity of accuracy to mesh quality supports Mueller’s position that poor cell quality is a stability issue. Accordingly, the approach with STAR-CCM+ is to be conservative – opt for robustness over accuracy. Specifically, they are looking for metrics that will result in division by zero in the solver. Skewness as it effects diffusion flux and linearization is one such example.

Mesher’s Perspective

Dr. John Steinbrenner and Nick Wyman shared Pointwise’s perspective on solution-independent quality metrics by taking a counter-intuitive approach. You would think that a mesh generation developer would promote the efficacy of a priori metrics. But the error in a CFD solution consists of geometric errors, discretization errors, and modeling errors. Geometric errors are similar to points made by Dannenhoffer and Mueller about properly representing the shape. Modeling errors come from turbulence, chemical, and thermophysical properties. Discretization involves degradation of the solver’s numerics. The discretization error is driven by coupling between the mesh and the solver’s numerical algorithm.

Figure 3: This table summarizes the mesh quality metrics available in Pointwise. From Reference 1h.

Therefore, although Pointwise can compute and display many metrics, it is important to note that many of them lack a direct relationship to the solver’s numerics and accordingly they are only loose indicators of solution accuracy. On the other hand, these metrics are convenient to compute, can address Dannenhoffer’s grid validity issue, and provide a mechanism for launching mesh improvement techniques. They also form the basis of a user’s ability to develop domain expertise – metrics that correlate to their specific application domain.

Conclusions

  1. CFD solver developers believe mesh quality affects convergence much more than accuracy. Therefore, the solution error due to poor or incomplete convergence cannot be ignored.
  2. One researcher was able to show a complete lack of correlation between mesh quality and solution accuracy. It would be valuable to reproduce this result for other solvers and flow conditions.
  3. Use as many grid points as possible (Dannenhoffer, McDaniel). In many cases, resolution trumps quality. However, the practical matter of minimizing compute time by using the minimum number of points (what Thornburg called an optimum mesh) means that quality still will be important.
  4. A priori metrics are valuable to users as an effective confidence check prior to running the solver. It is important that these metrics account for cell geometry but also the solver’s numerical algorithm. The implication is that metrics are solver-dependent. A further implication is that Dannehoffer’s grid validity checks be implemented.
  5. There are numerous quality metrics that can be computed, but they are often computed inconsistently from program to program. Development of a common vocabulary for metrics would aid portability.
  6. Interpreting metrics can be difficult because their actual numerical values are non-intuitive and stymie development of domain expertise. A metric vocabulary should account for desired range of result numerical values and the meaning of “bad” and “good.”

References

  1. Workshop presentations
    1. Stephen Alter, NASA Langley, “A Structured-Grid Quality Measure”
    2. John Dannehoffer, Syracuse University, “On Grid Quality and Validity”
    3. Christopher Roy, Virginia Tech, “Discretization Error”
    4. Vinit Gupta, Metacomp Technologies, “CFD++ Perspective on Mesh Quality”
    5. Konstantine Kourbataski, ANSYS, “Assessment of Mesh Quality in ANSYS CFD”
    6. David McDaniel, University of Alabama at Birmingham, “Kestrel/CREATE-AV Perspective on Mesh Quality”
    7. Alan Mueller, CD-adapco, “A CD-adapco Perspective on Mesh Quality”
    8. John Steinbenner and Nick Wyman, Pointwise, “Solution Independent Metrics”
    9. Presentations from the Mesh Quality Workshop are available by email request to pettt-requests@drc.com.
  2. Thornburg, Hugh J., “Overview of the PETTT Workshop on Mesh Quality/Resolution, Practice, Current Research, and Future Directions”, AIAA paper no. 2012-0606, Jan. 2012.
  3. Stimpson, C.J. et al, “The Verdict Geometric Quality Library”, Sandia Report 2007-1751, 2007.
  4. Mavriplis, Dimitri J., “Grid Quality and Resolution Issues from the Drag Prediction Workshop Series”, AIAA paper 2008-930, Jan. 2008.
  5. Roache, P.J., “Quantification of Uncertainty in Computational Fluid Dynamics” Annual Review of Fluid Mechanics Vol. 29, 1997, pp. 123-160.
  6. Knupp, Patrick M., “Remarks on Mesh Quality”, AIAA, Jan. 2007.

Subscribe to The Connector

Subscribe to The Connector for access to more articles like this one, application stories, and tips on generating meshes using Pointwise.

This entry was posted in Events, Software and tagged , , , , , , , , , , , , . Bookmark the permalink.

20 Responses to Accuracy, Convergence and Mesh Quality

  1. Well, this is a tough topic. Here is another example. Not sure if anyone mentioned this. One unfortunate aspect about non linear aerodyanmics is that multiple solutions can exists and how one gets to a solution is important. For example we use methods such as multi-grid and local time stepping to increase the rate of convergence for steady state runs, however, that does not mean that how one gets to the solution is actually physically possible. Of course the multi-grid or local time stepping behavior is dependent on the grid, but it is dependent in the sense that if the grid makes the method behave like a time accurate solution, then the grid is “good”. But that means the convergence is slow. Therefore a trade-off exists.

    http://hegedusaero.com/examples/ConeShoulder/ConeShoulder.html

  2. John Chawner says:

    If memory serves me correctly, it was at either the most recent High Lift or Drag workshop that the issue of starting point came up – if you start a simulation from scratch you get one answer but if you piggyback on a previous solution you get a different answer. So you’re right, a lot of the strategies we use to keep run time and convergence down my also be effecting accuracy.

  3. John Chawner says:

    Martin: Yes, perhaps that’s what I was recalling. However, Adobe Acrobat is so freakin’ slow that I may never find out for certain.

  4. Carl Ollivier-Gooch says:

    I stand by my statement. We can’t say anything beyond “Refinement helps accuracy” and “Good meshes converge better than bad meshes”. Not even a definition of what’s “good” and what’s “bad” beyond what the solver tells us at run time. This is especially true for unstructured meshes. After 30+ years of unstructured mesh finite volume methods in computational aero, I think that qualifies as knowing embarrassingly little.

  5. Pingback: CFD mesh quality: understanding accuracy and convergence | 3D CAD Tips

  6. John Chawner says:

    Carl: Does this at least qualify as knowing what we don’t know? Or are we still in the not knowing what we don’t know stage? In other words, admitting ignorance should put us on the path to enlightenment.

    • Carl Ollivier-Gooch says:

      John,

      I agree with Martin: we don’t yet know what we don’t know. About all we know for sure is that more resolution helps, and that solution-based adaptivity makes that cheaper. We have (at best) only anecdotal evidence about what matters in terms of mesh quality above some (pretty awful) threshold.

      Don’t worry, if I wake up in the middle of the night knowing the answer, I’ll write it down so I don’t forget, and call you first thing in the morning.

  7. Maybe we are still be in the “not knowing what we don’t know stage” And, IMO, someone needs to define “accuracy” in regards to a CFD solution before we can apply the term to grids.

    For example the “Post-Workshop Grid Studies” section of people.nas.nasa.gov/~pulliam/mypapers/AIAA-2010-4219.pdf. The ultra-fine grid had 2.4 billion points. I would guess very few entities can create solutions of this size. Other than grid density, what metrics apply to such a grid? Sure a grid cell could be skewed, but, with that density, would a good grid generator actually need to create a skewed (or bad) grid? And, how may solvers, (or unstructured grid generators?), can handle 2.4 billion points? And it is depressing (in terms of analyzing accuracy at the high fidelity level) to think that a flow feature, such as the wing body junction separation, disappears at these high grid densities (189 million pts) and that the trailing edge separation is changing enough to affect the solution on a global scale, even up to 2.4 billion (alpha (CL=0.5) changes from (about) 2.308 to 2.285 (1% diff) from grid densities of 213 million to 2.4 billion). However, it is not surprising, just depressing. And this is a simple geometry.

    Maybe two categories of accuracy need to be defined. Engineering accuracy and High Fidelity accuracy.

  8. John Chawner says:

    Martin: I would rename your second accuracy as Scientific Accuracy. I’m mostly worried about Engineering Accuracy in the sense of being able to provide tools that deliver results with a known error band. If someone else wants to work on techniques for driving CFD accuracy to ultra-high precision that’s great. Stated another way, maybe engineering accuracy means getting the trends right while scientific accuracy means nailing one single solution.

    The older I get the happier I am simply to be able to ask the right questions. It seems to me that the mesh quality workshop has us doing that.

  9. OK, Scientific Accuracy then since CFD as a whole is categorized as high fidelity.

    I am definitely on board with your last statement.

    It will also be interesting to see how grid metrics evolve to handle these massive cases. Personally, I don’t think I can adequately visualize what a 2.4 billion cell grid looks like in terms of grid quality. And, I guess, it is not so much about the individual cell, but about trends in cells (as was shown in the work shop)

  10. John Chawner says:

    Martin: Not to go off on a tangent but your comment about CFD being categorized as high fidelity may be part of our collective problem. (I am not pinning this problem on you, just saying that your comment made me think about it.) Government subsidies aside, maybe CFD needs to be like General Motors – sure there’s a Cadillac Escalade but most people could benefit from a Chevy Cavalier to get from place to place. CFD needs to be positioned as an engineering tool not some esoteric science for experts.

    Handling billion cell problems reminds me of something I read recently about how automated systems (the only way you’ll practically make a billion cell mesh) require more operator oversight than manual system. In other words, even though automatic means “without human intervention” the reverse is true in practice.

    Another random thought: if number of points trumps cell quality issues, why not just use the brute force approach and throw hundreds of millions of cells at problems? After all, that’s just a computer hardware issue (storage, RAM, processing). Stop wasting engineer’s time finessing things cuz the engineer’s time is worth way more than hardware.

  11. Following your tangent, it’s interesting that over the years (panel, Euler, and earlier NS/RANS codes) engineering and scientific accuracy were one and the same. In other words, in a way, a Cadillac Escalade didn’t exist. Now they are diverging. The Escalade model has been introduced. At least for this (airplane) type of problem. (I’m neglecting laminar to turbulent transition) However, other areas, such as base separation, engineering and scientific are still hand-in-hand. From what I understand, from others and personal experience, base drag using RANS can be 20-40% off. (Ford Model-T) However, DES/LES brings it down to a couple of percent, which is well within engineering accuracy. So, over the coming years, we’ll get there too.

    Yup, the brute force method has a prominent place in the process. And, over time, we’ll learn the ins and outs of it. After all, at a minimum, how we throw the points at a problem does affect the convergence (time stepping) rate (accuracy).

  12. What part did roundoff errors play in the large simulations? Back in my spectral methods days, roundoff errors got involved as the pseudospectral derivative increased in size (became more “accurate”).

  13. John Chawner says:

    Patrick: Sorry, I don’t know the answer to that question.

  14. I thought I would add more substance to my base drag and eddy viscosity comments by showing an analysis of a decelerator on my web site. http://www.hegedusaero.com/examples/Decelerator/Decelerator.html

  15. Pingback: Verification and Validation of CFD results | cfdrevolutions

  16. Pingback: Another Fine Year in Blogging About Meshing | Another Fine Mesh

  17. Pingback: How to simplify CAD model for FEA analysis?

  18. Pingback: This Year in CFD Blogging | Another Fine Mesh

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s