Skip to content

Optimization Opportunities

Jerry Morrison edited this page Mar 13, 2019 · 5 revisions

Potential opportunities to make the software run significantly faster and/or in less memory:

  • [Fixed in newer releases of Fireworks?] Fireworks commands like lpad get_fws can take 10-20 seconds (!), apparently loading and serializing the tasks (classes or instances?) in order to Register your Firetask, and at least one of those tasks is making it recurse over much of the codebase.
    It might save a bunch of time to register the firetasks another way: "modify the USER_PACKAGES variable of the FW config to include the package for where to find the Firetask..."
  • Jenkins builds can run out of memory.
  • Running all the analysis scripts can run out of memory. Are some of them leaking large objects?
    The Python garbage collector frees objects in reference cycles, but it won't free objects that have __del__ methods because it can't determine a safe order to destroy them.
    Here's a way to detect them:
    gc.set_debug(gc.DEBUG_UNCOLLECTABLE | gc.DEBUG_INSTANCES | gc.DEBUG_OBJECTS)

    ...  # run an analysis plot

    gc.collect()
    # gc.garbage is a list of objects the collector found to be unreachable but
    # couldn't free since they have __del__ methods.
    print "***DEBUG: gc.garbage=", gc.garbage
    gc.set_debug(0)
  • polymerize.py has more optimization opportunities.
  • mc_complexation.pyx might benefit from using memory views onto NumPy arrays instead of the older Cython access.