Skip to content

Unexpected instance dict size increase #210

Description

@Jongy

Since PEP 412 (landed in Python 3.3/3.4), Python has support for split dictionaries: values and keys can be kept separately to allow for multiple dictionaries (with different values) to share the same keys, reducing memory usage. This was implemented solely for dictionaries holding instance attributes (as of today Python doesn't use this trick anywhere else) and it greatly reduces the size of each instance.

This optimization is seamless, but some actions will disable it for a given type object:

  • Adding additional attributes after multiple instances have been created: for example, an__init__ method that adds 4 attributes, then another method adds 3 more. This unshares the particular dictionary, plus disables the optimization for newly created instances.
  • Deleting an attribute of an instance: del instance.a. This also disables the optimization for new instances.
  • Using non-str key in the instance __dict__ - this makes no sense though. For example instance.__dict__[1] = 1. This unshares the particular dictionary (doesn't disable for future instances)

The last case is not so interesting, but the first and second are not unimaginable, and they have a huge effect on memory consumption (can be demonstrated easily with sys.getsizeof or tracemalloc).

Metadata

Metadata

Assignees

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions