Effective Python

Master the art of Python programming with 125 actionable best practices to write more efficient, readable, and maintainable code.   Python is a versatile and powerful language, but leveraging its full potential requires more than just knowing the syntax. Effective Python: 125 Specific Ways to Write Better Python, 3rd Edition is your comprehensive guide to mastering Python's unique strengths and avoiding its hidden pitfalls. This updated edition builds on the acclaimed second edition, expanding from 90 to 125 best practices that are essential for writing high-quality Python code.   Drawing on years of experience at Google, Brett Slatkin offers clear, concise, and practical advice for both new and experienced Python developers. Each item in the book provides insight into the "Pythonic" way of programming, helping you understand how to write code that is not only effective but also elegant and maintainable. Whether you're building web applications, analyzing data, writing automation scripts, or training AI models, this book will equip you with the skills to make a significant impact using Python.   Key Features of the 3rd Edition: Expanded Content: Now with 125 actionable guidelines, including 35 entirely new items. Updated Best Practices: Reflects the latest features in Python releases up to version 3.13. New Chapters: Additional chapters on how to build robust programs that achieve high performance. Advanced Topics: In-depth coverage of creating C-extension modules and interfacing with native shared libraries. Practical Examples: Realistic code examples that illustrate each best practice.

Preface         xvii
Acknowledgments         xxiii
About the Author         xxv
 
Chapter 1: Pythonic Thinking         1
        Item 1: Know Which Version of Python Youre Using         1
        Item 2: Follow the PEP 8 Style Guide         3
        Item 3: Never Expect Python to Detect Errors at Compile Time         6
        Item 4: Write Helper Functions Instead of Complex Expressions         8
        Item 5: Prefer Multiple-Assignment Unpacking Over Indexing         11
        Item 6: Always Surround Single-Element Tuples with Parentheses         16
        Item 7: Consider Conditional Expressions for Simple Inline Logic         19
        Item 8: Prevent Repetition with Assignment Expressions         24
        Item 9: Consider match for Destructuring in Flow Control; Avoid When if Statements Are Sufficient         30
 
Chapter 2: Strings and Slicing         41
        Item 10: Know the Differences Between bytes and str         41
        Item 11: Prefer Interpolated F-Strings over C-Style Format Strings and str. format         47
        Item 12: Understand the Difference Between  repr and str when Printing Objects         58
        Item 13: Prefer Explicit String Concatenation over Implicit, Especially in Lists         62
        Item 14: Know How to Slice Sequences         67
        Item 15: Avoid Striding and Slicing in a Single Expression         70
        Item 16: Prefer Catch-All Unpacking Over Slicing         72
 
Chapter 3: Loops and Iterators         77
        Item 17: Prefer enumerate over range         77
        Item 18: Use zip to Process Iterators in Parallel         79
        Item 19: Avoid else Blocks After for and while Loops         82
        Item 20: Never Use for Loop Variables After the Loop Ends         85
        Item 21: Be Defensive when Iterating over Arguments         87
        Item 22: Never Modify Containers While Iterating over Them; Use Copies or Caches Instead         92
        Item 23: Pass Iterators to any and all for Efficient Short-Circuiting Logic         98
        Item 24: Consider itertools for Working with Iterators and Generators         102
 
Chapter 4: Dictionaries         109
        Item 25: Be Cautious when Relying on Dictionary Insertion Ordering         109
        Item 26: Prefer get over in and KeyError to Handle Missing Dictionary Keys         117
        Item 27: Prefer defaultdict over setdefault to Handle Missing Items in Internal State         122
        Item 28: Know How to Construct Key-Dependent Default Values with __missing__         124
        Item 29: Compose Classes Instead of Deeply Nesting Dictionaries, Lists, and Tuples         127
 
Chapter 5: Functions         135
        Item 30: Know That Function Arguments Can Be Mutated         135
        Item 31: Return Dedicated Result Objects Instead of Requiring Function Callers to Unpack More Than Three Variables         138
        Item 32: Prefer Raising Exceptions to Returning None         142
        Item 33: Know How Closures Interact with Variable Scope and nonlocal         145
        Item 34: Reduce Visual Noise with Variable Positional Arguments         150
        Item 35: Provide Optional Behavior with Keyword Arguments         153
        Item 36: Use None and Docstrings to Specify Dynamic Default Arguments         157
        Item 37: Enforce Clarity with Keyword-Only and Positional-Only Arguments         161
        Item 38: Define Function Decorators with functools. wraps         166
        Item 39: Prefer functools. partial over lambda Expressions for Glue Functions         169
 
Chapter 6: Comprehensions and Generators         173
        Item 40: Use Comprehensions Instead of map and filter         173
        Item 41: Avoid More Than Two Control Subexpressions in Comprehensions         176
        Item 42: Reduce Repetition in Comprehensions with Assignment Expressions         178
        Item 43: Consider Generators Instead of Returning Lists         182
        Item 44: Consider Generator Expressions for Large List Comprehensions         184
        Item 45: Compose Multiple Generators with yield from         186
        Item 46: Pass Iterators into Generators as Arguments Instead of Calling the send Method         188
        Item 47: Manage Iterative State Transitions with a Class Instead of the Generator throw Method         195
 
Chapter 7: Classes and Interfaces         201
        Item 48: Accept Functions Instead of Classes for Simple Interfaces         201
        Item 49: Prefer Object-Oriented Polymorphism over Functions with isinstance Checks         205
        Item 50: Consider functools. singledispatch for Functional-Style Programming Instead of Object-Oriented Polymorphism         210
        Item 51: Prefer dataclasses for Defining Lightweight Classes         217
        Item 52: Use @classmethod Polymorphism to Construct Objects Generically         230
        Item 53: Initialize Parent Classes with super         235
        Item 54: Consider Composing Functionality with Mix-in Classes         240
        Item 55: Prefer Public Attributes over Private Ones         245
        Item 56: Prefer dataclasses for Creating Immutable Objects         250
        Item 57: Inherit from collections. abc Classes for Custom Container Types         260
 
Chapter 8: Metaclasses and Attributes         265
        Item 58: Use Plain Attributes Instead of Setter and Getter Methods         265
        Item 59: Consider @property Instead of Refactoring Attributes         270
        Item 60: Use Descriptors for Reusable @property Methods         274
        Item 61: Use __getattr__, __getattribute__, and __setattr__ for Lazy Attributes         279
        Item 62: Validate Subclasses with __init_subclass__         285
        Item 63: Register Class Existence with __init_subclass__         293
        Item 64: Annotate Class Attributes with __set_name__         299
        Item 65: Consider Class Body Definition Order to Establish Relationships Between Attributes         303
        Item 66: Prefer Class Decorators over Metaclasses for Composable Class Extensions         310
 
Chapter 9: Concurrency and Parallelism         319
        Item 67: Use subprocess to Manage Child Processes         320
        Item 68: Use Threads for Blocking I/O; Avoid for Parallelism         324
        Item 69: Use Lock to Prevent Data Races in Threads         330
        Item 70: Use Queue to Coordinate Work Between Threads         333
        Item 71: Know How to Recognize When Concurrency Is Necessary         344
        Item 72: Avoid Creating New Thread Instances for On-Demand Fan-out         349
        Item 73: Understand How Using Queue for Concurrency Requires Refactoring         353
        Item 74: Consider ThreadPoolExecutor When Threads Are Necessary for Concurrency         361
        Item 75: Achieve Highly Concurrent I/O with Coroutines         364
        Item 76: Know How to Port Threaded I/O to asyncio         368
        Item 77: Mix Threads and Coroutines to Ease the Transition to asyncio         381
        Item 78: Maximize Responsiveness of asyncio Event Loops with async-friendly Worker Threads         389
        Item 79: Consider concurrent. futures for True Parallelism         393
 
Chapter 10: Robustness         399
        Item 80: Take Advantage of Each Block in try/except/else/finally         399
        Item 81: assert Internal Assumptions and raise Missed Expectations         404
        Item 82: Consider contextlib and with Statements for Reusable try/finally Behavior         408
        Item 83: Always Make try Blocks as Short as Possible         412
        Item 84: Beware of Exception Variables Disappearing         414
        Item 85: Beware of Catching the Exception Class         416
        Item 86: Understand the Difference Between Exception and BaseException         419
        Item 87: Use traceback for Enhanced Exception Reporting         424
        Item 88: Consider Explicitly Chaining Exceptions to Clarify Tracebacks         428
        Item 89: Always Pass Resources into Generators and Have Callers Clean Them Up Outside         436
        Item 90: Never Set __debug__ to False         442
        Item 91: Avoid exec and eval Unless Youre Building a Developer Tool         445
 
Chapter 11: Performance         447
        Item 92: Profile Before Optimizing         448
        Item 93: Optimize Performance-Critical Code Using timeit Microbenchmarks         453
        Item 94: Know When and How to Replace Python with Another Programming Language         458
        Item 95: Consider ctypes to Rapidly Integrate with Native Libraries         462
        Item 96: Consider Extension Modules to Maximize Performance and Ergonomics         467
        Item 97: Rely on Precompiled Bytecode and File System Caching to Improve Startup Time         475
        Item 98: Lazy-Load Modules with Dynamic Imports to Reduce Startup Time         478
        Item 99: Consider memoryview and bytearray for Zero-Copy Interactions with bytes         485
 
Chapter 12: Data Structures & Algorithms         493
        Item 100: Sort by Complex Criteria Using the key Parameter         493
        Item 101: Know the Difference Between sort and sorted         499
        Item 102: Consider Searching Sorted Sequences with bisect         501
        Item 103: Prefer deque for Producer-Consumer Queues         504
        Item 104: Know How to Use heapq for Priority Queues         509
        Item 105: Use datetime Instead of time for Local Clocks         519
        Item 106: Use decimal When Precision Is Paramount         523
        Item 107: Make pickle Serialization Maintainable with copyreg         526
 
Chapter 13: Testing and Debugging         533
        Item 108: Verify Related Behaviors in TestCase Subclasses         533
        Item 109: Prefer Integration Tests over Unit Tests         541
        Item 110: Isolate Tests From Each Other with setUp, tearDown, setUpModule, and tearDownModule         547
        Item 111: Use Mocks to Test Code with Complex Dependencies         550
        Item 112: Encapsulate Dependencies to Facilitate Mocking and Testing         559
        Item 113: Use assertAlmostEqual to Control Precision in Floating Point Tests         563
        Item 114: Consider Interactive Debugging with pdb         565
        Item 115: Use tracemalloc to Understand Memory Usage and Leaks         570
 
Chapter 14: Collaboration         575
        Item 116: Know Where to Find Community-Built Modules         575
        Item 117: Use Virtual Environments for Isolated and Reproducible Dependencies         576
        Item 118: Write Docstrings for Every Function, Class, and Module         582
        Item 119: Use Packages to Organize Modules and Provide Stable APIs         588
        Item 120: Consider Module-Scoped Code to Configure Deployment Environments         593
        Item 121: Define a Root Exception to Insulate Callers from APIs         595
        Item 122: Know How to Break Circular Dependencies         600
        Item 123: Consider warnings to Refactor and Migrate Usage         605
        Item 124: Consider Static Analysis via typing to Obviate Bugs         613
        Item 125: Prefer Open Source Projects for Bundling Python Programs over zipimport and zipapp         621
 
Index         627