Project: pasta (GitHub Link)

pasta-master
- testdata
  - ast
    - num.in
    - dictcomp.in
    - yield.in
    - namedexpr.in
    - subscript.in
    - lambda.in
    - golden
      - 3.8
        boolop.out
        for.out
        generators.out
        assign.out
        async.out
        functiondef3.out
        fstring.out
        call.out
        subscript.out
        raise3.out
        compexps.out
        dictcomp.out
        binop3.out
        lambda.out
        classdef.out
        functiondef38.out
        del.out
        binop.out
        augassign.out
        num3.out
        set.out
        semicolons.out
        dict.out
        yieldfrom.out
        list.out
        with.out
        attribute.out
        ifexp.out
        functiondef.out
        if.out
        empty.out
        str.out
        import.out
        trailing_comments.out
        try.out
        types.out
        dict3.out
        num.out
        unaryop.out
        ellipsis.out
        constant.out
        classdef3.out
        slice.out
        fromimport.out
        assert.out
        while.out
        nonlocal.out
        bytes.out
        global.out
        yield.out
        annassign.out
        compare.out
        tuple.out
      - 3.4
        boolop.out
        for.out
        generators.out
        assign.out
        functiondef3.out
        call.out
        subscript.out
        raise3.out
        compexps.out
        dictcomp.out
        lambda.out
        classdef.out
        del.out
        binop.out
        augassign.out
        set.out
        semicolons.out
        dict.out
        yieldfrom.out
        list.out
        with.out
        attribute.out
        ifexp.out
        functiondef.out
        if.out
        empty.out
        str.out
        import.out
        trailing_comments.out
        try.out
        types.out
        num.out
        unaryop.out
        sample.out
        ellipsis.out
        constant.out
        classdef3.out
        slice.out
        fromimport.out
        assert.out
        while.out
        nonlocal.out
        bytes.out
        global.out
        yield.out
        compare.out
        tuple.out
      - 2.7
        boolop.out
        for.out
        exec.out
        generators.out
        assign.out
        call.out
        subscript.out
        try2.out
        print.out
        compexps.out
        dictcomp.out
        lambda.out
        classdef.out
        del.out
        binop.out
        augassign.out
        set.out
        raise.out
        semicolons.out
        dict.out
        compare2.out
        list.out
        with.out
        attribute.out
        functiondef2.out
        ifexp.out
        functiondef.out
        if.out
        empty.out
        str.out
        import.out
        repr.out
        trailing_comments.out
        bom.out
        try.out
        num.out
        unaryop.out
        sample.out
        slice.out
        fromimport.out
        assert.out
        while.out
        bytes.out
        global.out
        yield.out
        compare.out
        tuple.out
      - 3.5
        boolop.out
        for.out
        generators.out
        assign.out
        functiondef3.out
        call.out
        subscript.out
        raise3.out
        compexps.out
        dictcomp.out
        binop3.out
        lambda.out
        classdef.out
        del.out
        binop.out
        augassign.out
        set.out
        semicolons.out
        dict.out
        yieldfrom.out
        list.out
        with.out
        attribute.out
        ifexp.out
        functiondef.out
        if.out
        empty.out
        str.out
        import.out
        trailing_comments.out
        try.out
        types.out
        dict3.out
        num.out
        unaryop.out
        ellipsis.out
        constant.out
        classdef3.out
        slice.out
        fromimport.out
        assert.out
        while.out
        nonlocal.out
        bytes.out
        global.out
        yield.out
        compare.out
        tuple.out
      - 3.7
        boolop.out
        for.out
        generators.out
        assign.out
        async.out
        functiondef3.out
        fstring.out
        call.out
        subscript.out
        raise3.out
        compexps.out
        dictcomp.out
        binop3.out
        lambda.out
        classdef.out
        del.out
        binop.out
        augassign.out
        num3.out
        set.out
        semicolons.out
        dict.out
        yieldfrom.out
        list.out
        with.out
        attribute.out
        ifexp.out
        functiondef.out
        if.out
        empty.out
        str.out
        import.out
        trailing_comments.out
        try.out
        types.out
        dict3.out
        num.out
        unaryop.out
        ellipsis.out
        constant.out
        classdef3.out
        slice.out
        fromimport.out
        assert.out
        while.out
        nonlocal.out
        bytes.out
        global.out
        yield.out
        annassign.out
        compare.out
        tuple.out
      - 3.6
        boolop.out
        for.out
        generators.out
        assign.out
        functiondef3.out
        fstring.out
        call.out
        subscript.out
        raise3.out
        compexps.out
        dictcomp.out
        binop3.out
        lambda.out
        classdef.out
        del.out
        binop.out
        augassign.out
        num3.out
        set.out
        semicolons.out
        dict.out
        yieldfrom.out
        list.out
        with.out
        attribute.out
        ifexp.out
        functiondef.out
        if.out
        empty.out
        str.out
        import.out
        trailing_comments.out
        try.out
        types.out
        dict3.out
        num.out
        unaryop.out
        ellipsis.out
        constant.out
        classdef3.out
        slice.out
        fromimport.out
        assert.out
        while.out
        nonlocal.out
        bytes.out
        global.out
        yield.out
        annassign.out
        compare.out
        tuple.out
    - del.in
    - bytes.in
    - exec.in
    - unaryop.in
    - slice.in
    - ifexp.in
    - binop.in
    - semicolons.in
    - fstring.in
    - augassign.in
    - tuple.in
    - assert.in
    - functiondef2.in
    - dict.in
    - raise.in
    - functiondef.in
    - num3.in
    - empty.in
    - trailing_comments.in
    - while.in
    - yieldfrom.in
    - ellipsis.in
    - raise3.in
    - constant.in
    - compexps.in
    - functiondef38.in
    - str.in
    - boolop.in
    - binop3.in
    - try.in
    - fromimport.in
    - for.in
    - nonlocal.in
    - if.in
    - global.in
    - classdef.in
    - generators.in
    - types.in
    - call.in
    - import.in
    - repr.in
    - attribute.in
    - try2.in
    - print.in
    - bom.in
    - list.in
    - compare2.in
    - functiondef3.in
    - compare.in
    - set.in
    - classdef3.in
    - dict3.in
    - async.in
    - assign.in
    - annassign.in
    - with.in
  - codegen
    - imports.in
    - classes.in
    - values.in
    - del.in
    - assert.in
    - raise.in
    - try.in
    - if.in
    - loops.in
    - assignments.in
    - index.in
    - attribute.in
    - print.in
    - functions.in
    - with.in
- LICENSE
- pasta
  - augment
    - import_utils.py
    - inline.py
    - import_utils_test.py
    - inline_test.py
    - __init__.py
    - errors.py
    - rename.py
    - rename_test.py
  - base
    - test_utils.py
    - ast_utils_test.py
    - scope_test.py
    - formatting.py
    - ast_utils.py
    - test_utils_test.py
    - token_generator.py
    - codegen_test.py
    - codegen.py
    - scope.py
    - __init__.py
    - annotate_test.py
    - fstring_utils.py
    - ast_constants.py
    - annotate.py
  - __init__.py
- CONTRIBUTING.md
- setup.py
- .travis.yml
- README.md
- .gitignore

pasta: Python AST Augmentation

This is still a work-in-progress; there is much more to do. Existing functionality may not be perfect.

Mission

Enable python source code refactoring through AST modifications.

Sample use cases:

Facilitate moving or renaming python modules by rewriting import statements.
Refactor code to enforce a certain style, such as reordering function definitions.
Safely migrate code from one API to another.

Design Goals

Symmetry: Given any input source, it should hold that pasta.dump(pasta.parse(src)) == src.
Mutability: Any changes made in the AST are reflected in the code generated from it.
Standardization: The syntax tree parsed by pasta will not introduce new nodes or structure that the user must learn.

Python Version Support

Supports python 2.7 and up to 3.8.

Dependencies

pasta depends on six.

Basic Usage

import pasta
tree = pasta.parse(source_code)

# ... Augment contents of tree ...

source_code = pasta.dump(tree)

Built-in Augmentations

Pasta includes some common augmentations out-of-the-box. These can be used as building blocks for more complex refactoring actions.

There will be more of these basic augmentations added over time. Stay tuned!

Rename an imported name

Rewrites references to an imported name, module or package. For some more examples, see pasta/augment/rename_test.py.

# Rewrite references from one module to another
rename.rename_external(tree, 'pkg.subpkg.module', 'pkg.other_module')

# Rewrite references from one package to another
rename.rename_external(tree, 'pkg.subpkg', 'pkg.other_pkg')

# Rewrite references to an imported name in another module
rename.rename_external(tree, 'pkg.module.Query', 'pkg.module.ExecuteQuery')

Known issues and limitations

Changing the indentation level of a block of code is not supported. This is not an issue for renames, but would cause problems for refactors like extracting a method.
pasta works under the assumption that the python version that the code is written for and the version used to run pasta are the same. This is because pasta relies on ast.parse
Some python features are not fully supported, including global.

Developing

This project uses setuptools to facilitate testing and packaging.

# Run all tests
python setup.py test

# Run a single test suite
python setup.py test -s pasta.base.annotate_test.suite

Disclaimer

This is not an official Google product.