The with statement (and by extension, async with) is all about resource management. Let me break it down:
- Basic
withexample first:
# Without 'with'file = open('file.txt')try: data = file.read()finally: file.close()
# With 'with' - same thing but cleanerwith open('file.txt') as file: data = file.read()# File automatically closes when we exit the with block- The
async withfollows the same principle but for asynchronous resources:
# Without async withactor = await Actor.initialize()try: # do stuff with actor await actor.cleanup()finally: await actor.final_cleanup()
# With async with - same thing but cleanerasync with Actor: # do stuff with actor# Actor automatically cleans up when we exit the blockThink of with like a sandwich:
- Opening the
withblock is like the top slice of bread - Your code is the filling
- Closing the
withblock is like the bottom slice of bread
The with statement ensures cleanup happens even if your code raises an exception - just like the bread holds the sandwich together even if some filling falls out.
In your Apify code:
async def main() -> None: async with Actor: # Actor starts up actor_input = await Actor.get_input() or {} # ... more code ... process.start() # Actor automatically shuts down properly hereThe async with Actor ensures that:
- The Actor properly initializes at the start
- Your code runs in the middle
- The Actor properly cleans up at the end, even if your code crashes
Multiple examples of both with and async with to help clarify their usage:
- Database Connection Example:
# Without 'with'connection = create_connection()try: cursor = connection.cursor() cursor.execute("SELECT * FROM users") results = cursor.fetchall()finally: connection.close()
# With 'with' - much cleanerwith create_connection() as connection: cursor = connection.cursor() cursor.execute("SELECT * FROM users") results = cursor.fetchall()- Multiple Resources Example:
# You can manage multiple resources in one with statementwith open('input.txt') as input_file, open('output.txt', 'w') as output_file: data = input_file.read() output_file.write(data.upper())- Async Database Example:
import asyncpg
async def get_users(): # Async database connection async with asyncpg.create_pool(database='mydb') as pool: async with pool.acquire() as connection: async with connection.transaction(): # Do database operations return await connection.fetch('SELECT * FROM users')- Custom Context Manager Example:
from contextlib import contextmanagerimport time
@contextmanagerdef timer(): start = time.time() yield # This is where the code inside the 'with' block runs end = time.time() print(f"Operation took {end - start} seconds")
# Using itwith timer(): # Some time-consuming operation time.sleep(2)- Async Resource Lock Example:
import asyncio
async def worker(lock, number): async with lock: # Only one worker can enter at a time print(f"Worker {number} starting...") await asyncio.sleep(1) print(f"Worker {number} finished!")
async def main(): lock = asyncio.Lock() workers = [worker(lock, i) for i in range(3)] await asyncio.gather(*workers)- Custom Async Context Manager:
from contextlib import asynccontextmanager
@asynccontextmanagerasync def managed_resource(): print("Starting up") try: yield "resource" # This is what the 'as' variable receives finally: print("Shutting down")
async def use_resource(): async with managed_resource() as resource: print(f"Using {resource}")- Real-world Redis Example:
import aioredis
async def cache_operation(): async with aioredis.from_url('redis://localhost') as redis: # Set value await redis.set('key', 'value') # Get value value = await redis.get('key') return value- HTTP Session Example:
import aiohttp
async def fetch_data(urls): async with aiohttp.ClientSession() as session: tasks = [] for url in urls: async with session.get(url) as response: data = await response.text() tasks.append(data) return tasksThese context managers are handling important setup and cleanup tasks like:
- Opening and closing files
- Starting and committing/rolling back database transactions
- Acquiring and releasing locks
- Opening and closing network connections
- Resource cleanup
- Performance measurement
- Error handling
The beauty of with and async with is that they guarantee proper resource management even if exceptions occur in your code. This prevents resource leaks and ensures clean shutdown of your applications.